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Executive  Summary 


This  post  doctoral  research  project  was  aimed  at  following  up  on  a  dissertation  projeet  by 
Dr.  Allison  Tollner  in  whieh  she  investigated  ehange  blindness  in  individuals  and  teams.  The 
primary  goal  was  to  examine  the  role  of  attention  and  eye  movements  on  the  ability  of  individual 
and  multiple  observers  to  detect  changes  in  a  complex  visual  seene.  Related  to  this  goal,  the 
project  involved  the  set  up  of  an  eye  traeking  laboratory  whieh  allowed  simultaneous  recording 
of  eye  movements  from  dyads.  A  seeondary  goal  was  to  determine  the  viability  of  the  Nearest 
Neighbor  Index,  an  analysis  of  point  dispersion  in  space,  as  a  physiological  measure  of  cognitive 
workload  in  a  ehange  detection  task. 

Change  blindness  results  from  the  absenee  of  attention  from  the  source  of  a  ehange. 
Unsurprisingly,  ehange  blindness  worsens  in  dual  task  and  high  workload  eonditions.  Such  is 
the  environment  in  whieh  teams  of  operators  in  military  command  and  control  missions  operate, 
yet  little  is  known  about  ehange  blindness  suseeptibility  under  these  eonditions.  A  flieker  task 
required  individuals  and  dyads  to  deteet  ehanges  under  easy  and  diffieult  task  eonditions. 
Response  time,  accuraey  of  responses,  eye  movements,  heart  rate,  and  subjective  workload 
measures  were  recorded  and  analyzed. 

Tollner’ s  findings  indicated  that  ehange  detection  performance  only  benefits  from  having 
more  eyes  or  resourees  when  observers  are  allowed  to  communicate  and  thereby  coordinate  their 
actions  and  strategies.  Results  of  the  present  study  replieate  these  findings  by  showing  that 
eommunieating  dyads  were  more  effieient  at  change  detection  than  individuals  and  non¬ 
communicating  dyads.  Our  analyses  of  eye  movements  provide  support  for  the  importanee  of 
eommunieation  in  mitigating  ehange  blindness  suseeptibility  in  teams  of  observers. 

Overall,  sean  paths  and  eye  movement  data  show  differenees  in  attention  allocation  and 
scan  strategies  among  individuals,  communicating,  and  non-communicating  dyads.  Our  data 
show  that  communicating  dyads  functioned  as  a  team,  with  several  major  eomponents  of 
teamwork  evident  in  their  sean  paths.  This  likely  eontributed  to  more  efficient  ehange  detection 
strategies  and  therefore,  better  performanee  overall.  In  contrast,  non-communicating  dyads 
appeared  to  depend  on  slow  and  ineffieient  seareh  strategies  in  comparison  to  individual 
observers  and  communicating  teams.  In  addition,  eommunieating  dyads  are  just  as  blind  to  no 
change  as  individual  observers  and  non-eommunieating  dyads.  All  groups,  regardless  of  ability 
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to  communicate  or  number  of  eyes  on  task,  were  slowed  to  determine  the  absenee  of  ehange  in 
the  visual  scene  and  yielded  less  efficient  eye  movement  patterns  on  trials  in  whieh  no  change 
oceurred  eompared  to  those  in  which  a  change  did  oecur.  Such  behaviors  could  delay  decision¬ 
making  in  environments  such  as  tactical  command  and  control  and  lead  to  eatastrophic 
outcomes.  These  findings  have  implications  for  adaptive  aids  and  tasks  sueh  as  tactical 
command  and  control  in  which  teams  of  operators  depend  on  a  common  and  current 
understanding  of  the  battlespaee  for  decision-making. 

Results  of  the  Nearest  Neighbor  Index  (NNI)  analysis  suggest  that  the  NNI  may  be  a 
viable  measure  of  mental  workload  for  ehange  detection  paradigms,  but  within  limits.  First, 
given  that  NNI  is  based  on  an  average  distance  between  points,  a  major  limitation  of  the  measure 
is  the  necessity  for  enough  points  for  accurate  and  reliable  NNIs.  This  limitation  may  preelude 
the  use  of  NNI  to  measure  workload  in  time  limited  tasks.  In  the  present  study,  fixations  from 
multiple  trials  and  change  magnitude  conditions  were  combined  to  overcome  this  problem  and 
obtain  a  reliable  NNI.  However,  the  use  of  NNI  may  be  eonstrained  in  environments  sueh  as 
tactical  C2  in  which  workload  may  vary  within  relatively  short  periods  of  time  yielding  only  few 
fixations  for  NNI  ealculation.  Second,  our  experimental  design  required  the  randomization  of 
icon  positions  which  confounded  the  caleulation  of  the  NNI.  In  the  easy  condition,  fewer  stimuli 
were  scattered  in  a  larger  space  whieh  resulted  in  greater  variability  in  distances  between  points 
in  the  easy  compared  to  the  difficult  condition.  Consequently,  the  average  distance  between 
points  was  eloser  to  random  in  the  easy  compared  to  the  diffieult  condition,  leading  to  higher 
NNIs  in  the  easy  eompared  to  the  difficult  condition.  This  is  counter  to  previous  findings  of 
higher  NNI  values  with  more  difficult  tasks  and  our  results  of  poorer  performance  in  the  difficult 
compared  to  the  easy  condition. 

Two  regression  models  were  developed  to  determine  the  best  predictors  of  change 
deteetion  performance  of  dyads. 
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Introduction 


Change  Blindness  (CB)  is  the  laek  of  awareness  of  considerable  changes  in  a  visual  scene 
when  attention  is  diverted  from  the  source  of  the  change  by  a  simultaneously  occurring 
event,  e.g.,  screen  flashes  (Rensink,  O’Regan  &  Clark,  1997).  CB  results  from  the  absence 
of  visual  attention  from  the  source  of  the  change  (Simons  &  Levin,  1997).  Unsurprisingly, 
CB  has  been  shown  to  increase  when  two  or  more  tasks  are  performed  simultaneously 
(McCarley,  Vais,  Pringle,  Kramer,  Irwin,  &  Strayer,  2004),  and  when  a  secondary  task 
imposes  high  demands  on  attention  (Muthard  &  Wickens,  2003).  One  domain  that  may  be 
particularly  relevant  to  the  CB  phenomenon  is  tactical  command  and  control  (C2).  This  task 
requires  teams  to  monitor  and  maintain  shared  awareness  of  the  tactical  battlespace  and 
immediately  detect  and  communicate  critical  changes,  e.g.  sudden  disappearance  of  aircraft 
from  the  display.  Failure  to  do  so  could  prove  catastrophic. 

Since  CB  occurs  due  to  an  observer’s  inattention  to  a  change  source,  it  could  be  argued 
that  multiple  observers  would  be  less  susceptible  to  CB  than  individual  observers. 

Essentially,  more  eyes  on  a  scene  should  equal  more  resources  that  may  be  allocated  to  the 
task.  Consistent  with  this  idea,  Tollner,  Riley,  Nelson,  Shockley,  and  Cummins-Sebree 
(2006)  found  that  groups  of  observers  were  less  susceptible  to  CB  than  individuals,  but  only 
if  they  communicated  verbally.  Tollner  and  her  colleagues  used  a  flicker  task  in  which  they 
manipulated  task  difficulty  and  the  ability  of  observers  to  verbally  communicate.  Results  of 
that  study  suggest  that  having  more  eyes  or  resources  for  change  detection  are  only  beneficial 
if  there  is  a  mechanism  such  as  verbal  communication  for  coordinating  the  allocation  and 
application  of  those  resources,  thereby  allowing  multiple  observers  to  function  as  a  team 
(Dickinson  &  McIntyre,  1997).  Such  a  conclusion  may  be  predicated  with  direct  evidence 
from  visual  scanning  behavior.  However,  since  Tollner’ s  methodology  did  not  include  eye 
movements,  the  effects  of  communication  on  attention  allocation  and  scanning  strategies  in 
teams  are  unknown. 

The  present  study  was  designed  to  extend  the  work  of  Tollner  et  al.  (2006)  by 
investigating  attention  allocation  and  the  role  of  scanning  strategies  in  CB  susceptibility  in 
individuals  and  multiple  observers.  We  examined  attention  allocation  through  fixations  and 
strategies  through  scanning  patterns.  Additionally,  we  sought  to  examine  the  effectiveness  of 
the  Nearest  Neighbor  Index  (NNI)  in  serving  as  a  physiological  measure  of  mental  workload. 
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Attention  allocation  was  examined  by  analysis  of  the  number  of  fixations  required  before 
a  response  was  made.  This  measure  was  based  on  two  assumptions:  saccadic  eye 
movements  executed  to  foveate  stimuli  (1)  precede  fixations,  and  (2)  are  obligatorily  linked 
with  shifts  of  attention  (Shephard,  Findlay,  &  Hockey,  1987).  However,  since  attention  may 
be  shifted  in  the  absence  of  saccades,  i.e.,  more  than  one  attentional  shift  may  occur  during  a 
fixation  (Posner,  1980;  Shephard,  et  ah,  1987),  the  measure  is  merely  an  estimate  of  shifts  or 
re-allocation  of  attention  during  a  task. 

Scanning  efficiency  was  measured  by  analysis  of  the  number  of  trials  with  re-scanning  or 
cross-scanning.  A  re-scanned  trial  was  a  trial  in  which  (1)  fixations  landed  on  a  stimulus 
more  than  once,  or  (2)  fell  within  1  degree  (°)  of  visual  angle  of  each  other,  and  (3)  two  or 
more  scan  paths  fell  within  1°  of  visual  angle  of  each  other.  The  1°  of  visual  angle  re-scan 
threshold  was  selected  based  on  evidence  that  the  attentional  focus,  the  functional  region  of 
the  visual  field  wherein  information  is  selected  for  immediate  perceptual  processing,  may  be 
scaled  to  the  size  of  relevant  stimuli  (Castiello  &  Umilta,  1990;  Eriksen  &  Yeh,  1985).  In  the 
present  study,  stimuli  were  approximately  1°  of  visual  angle  in  size  and  their  movements 
were  either  .5°  or  1.5°  of  visual  angle,  or  1°  on  average.  A  cross-scanned  trial  was  a  trial  in 
which  scan  paths  of  partners  in  a  dyad  overlapped. 

Re-scanning  and  cross-scanning  are  based  on  the  re-checking  hypothesis  proposed  by 
Treisman  and  Gelade  (1980)  to  account  for  findings  of  longer  search  times  in  visual  search 
when  the  specified  target  was  absent  compared  to  when  it  was  present  in  the  array.  They 
argued  that,  on  target  absent  trials,  people  not  only  search  the  stimulus  set  exhaustively,  they 
also  re-check  stimuli  for  confirmation  of  their  identities.  Consequently,  this  inefficient 
search  strategy  slowed  responses  on  target  absent  trials.  Similarly,  we  assumed  that  a  greater 
proportion  of  re-scanned  and  cross-scanned  trials  represented  an  inefficient  search  for 
change,  or  change  detection  strategy. 

A  secondary  goal  of  this  study  was  to  assess  the  efficacy  of  the  NNI  as  a  physiological 
measure  of  cognitive  workload.  Workload  is  defined  as  the  mental  effort  or  cognitive 
processing  resources  required  for  performing  a  task  (Norman  &  Bobrow,  1975),  with 
difficult  tasks  typically  leading  to  high  workload  by  placing  higher  demands  on  a  finite 
amount  of  cognitive  processing  resources.  Eye  movements  have  been  used  to  measure 
workload  with  varying  results  (reviewed  in  DiNocera,  Terenzi,  &  Camilli,  2006).  In  the 
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present  study,  we  used  NNI  to  relate  fixation  distribution  to  workload  ehanges.  NNI  is 
derived  by  comparing  the  mean  distance  between  actual  points  (e.g.,  fixations)  within  a 
specified  area  to  the  mean  distance  given  a  theoretical  random  distribution  of  the  same 
number  of  points  in  an  equal  area  (Clark  &  Evans,  1954).  Given  that  NNI  is  the  ratio  of  the 
average  distance  between  points  in  an  actual  and  a  theoretical  random  distribution,  the  more 
random  the  distribution,  the  closer  the  NNI  value  will  be  to  one. 

Recently,  DiNocera,  Camilli  &  Terenzi  (2007)  used  the  NNI  to  measure  pilot  workload, 
and  confirmed  previous  results  from  DiNocera,  Terenzi,  &  Camilli  (2006)  that  NNI,  thus, 
fixation  randomness,  increased  with  workload.  Consistency  across  two  different  tasks  -  one 
used  static  and  one  used  dynamic  displays  -  suggests  that  NNI  may  provide  a  domain- 
independent  workload  measure.  NNI  may  be  calculated  easily,  frequently  during  a  task,  and 
with  data  from  affordable,  non-invasive  technology,  making  it  accessible  to  many 
researchers.  Therefore,  the  viability  of  NNI  as  a  measure  of  workload  should  be  further 
investigated. 

CB  has  been  shown  to  increase  with  workload  (Muthard  &  Wickens,  2003),  which  varies 
significantly  during  C2  tasks  such  as  Air  Battle  Management  (Knott,  Bolia,  Nelson,  & 
Galster,  2006).  Where  catastrophic  outcomes  may  follow  workload-related  performance 
degradations,  timely  assessment  of  workload  may  allow  for  (a)  determination  of  changes  in 
task  distribution  within  teams,  or  (b)  triggering  adaptive  automation  aids  to  curtail 
performance  degradations  (Parasuraman,  2003).  However,  most  workload  measures  require 
data  to  be  gathered  over  time  to  determine  physiological  and  behavioral  responses  to  changes 
in  task  requirements,  or  a  task  is  interrupted  to  administer  subjective  measures  such  as  the 
National  Aeronautics  and  Space  Administration  Task  Load  Index  (NASA  TLX:  Hart  & 
Staveland,  1988).  Additionally,  both  physiological  and  behavioral  measures  have  provided 
conflicting  results  across  tasks.  Therefore,  multiple  workload  measures  are  recommended 
for  confirmation  of  results  within  a  study  (Wickens,  Gordon,  &  Liu,  1998). 

In  the  present  study,  we  examined  the  efficacy  of  NNI  as  an  immediate  and  reliable 
workload  measure.  To  this  end,  we  analyzed  heart  rate,  and  subjective  TLX  ratings  in 
conjunction  with  NNI  for  the  purpose  of  comparing  the  results  for  consistency.  Heart  rate  is 
the  most  widely  used  cardiac  measure  of  workload  and  has  been  shown  to  reliably  increase 
with  workload  (e.g.,  Wilson  &  Eggemeier,  1991;  Wilson,  2002).  Subjective  measures  such 
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as  the  NASA  TLX  are  commonly  used  in  applied  science  settings  (e.g.,  air  traffic 
management),  and  has  also  been  shown  to  reliably  increase  with  workload. 

In  addition,  we  used  the  Team  Workload  Scale  (TWS:  Hildebrand,  Pharmer,  &  Weaver, 
2003)  to  obtain  a  subjective  measure  of  the  workload  experienced  by  a  team  of  observers  and 
to  understand  the  role  of  communication  on  subjective  workload.  Given  that  individual 
workload  in  is  defined  by  task  demands  on  a  limited  amount  of  processing  resources,  team 
workload  is  defined  as  demands  of  a  task  on  a  limited  amount  of  resources  available  from  a 
team  of  individuals  to  perform  both  individual  and  team-related  tasks  such  as  communicating 
and  coordinating  actions  (Bowers,  Braun,  &  Morgan,  1997).  However,  the  TWS  has  not 
been  experimentally  validated  and  its  reliability  is  unknown.  This  questionnaire  was  used  in 
this  study  (a)  to  assess  the  contributions  of  verbal  communication  in  a  change  detection  task 
with  multiple  observers  and  (b)  to  maintain  consistency  between  methods  used  in  the  present 
study  and  that  of  Tollner  et  al  (2006). 

We  investigated  how  attention  allocation  and  visual  scanning  strategies  in  change 
detection  vary  when:  (1)  individuals  and  teams  perform  the  task;  (2)  communication  is 
allowed  or  prohibited;  and  (3)  task  difficulty  is  high  or  low.  Additionally,  we  compared 
results  of  the  NNI  measure  to  other  measures  of  workload  to  assess  its  effectiveness  in 
capturing  changes  in  workload  on  a  change  detection  task. 
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Methods 


Participants 

Twenty  participants  were  paid  $  15/hr  for  participation,  and  tested  as  individual  observers 
and  as  part  of  a  dyad.  At  the  beginning  of  each  session,  each  participant  was  required  to 
complete  an  informed  consent  form  and  a  biographical  questionnaire  (see  Appendix  A). 

Only  those  who  participated  in  both  the  individual  and  dyad  conditions  were  included  in  the 
final  analysis  (3  males  &  6  females  with  mean  age  of  23.4  years).  Participants  were  excluded 
based  on  two  criteria:  (1)  inability  to  return  for  subsequent  testing;  (2)  eye  tracker  calibration 
difficulties  due  to  sleepiness  or  glossiness  of  eyes.  All  but  one  participant’s  performance 
data  fell  within  2.5  SD  from  the  mean  -  participant  Q’s  average  accuracy  for  the  large  change 
set  size  48  condition  fell  at  2.66  SD  from  the  mean.  Results  from  ANOVAs  with  and 
without  participant  Q’s  accuracy  data  were  compared  and  all  but  one  interaction  were  the 
same.  The  ANOVA  comparing  individual  observers  and  communicating  dyads  without 
participant  Q’s  data  yielded  a  set  size  x  group  interaction  which  was  only  marginally 
significant  (p  =  .07)  with  participant  Q’s  data  included  in  the  analysis  (see  Appendix  D  for 
analysis  of  accuracy  from  individuals  and  communicating  dyads  with  and  without  participant 
Q).  Given  the  minimal  difference  in  the  results  with  and  without  Q’s  data  and  the  benefits  of 
including  Q’s  data  from  both  the  individual  observer  and  dyad  conditions,  participant  Q’s 
data  were  included  in  all  analyses. 

Participants  were  tested  as  individuals  and  as  part  of  a  dyad  in  different  sessions,  in 
counterbalanced  order.  As  shown  in  Table  1,  all  but  1  of  the  participants  included  in  the 
analyses  were  tested  twice  in  the  dyad  condition,  each  time  with  a  different  partner  having 
the  same  counterbalance  assignment  and  experience  with  the  task.  Participant  Q  was  tested 
only  once  as  part  of  a  dyad.  A  total  of  1 1  dyads  were  tested  in  both  a  Communication  and  a 
No  communication  condition.  Two  dyads  were  excluded  due  to  inability  of  a  member  of  the 
dyad  to  return  for  testing  in  the  Individual  observer  condition.  One  dyad  was  excluded  due  to 
eye  movement  recording  error  in  which  data  were  lost.  Also  shown  in  Table  1,  of  the 
remaining  participants  whose  data  were  included  in  the  analyses,  half  were  tested  first  as 
individuals  and  half  were  tested  last  as  individuals.  A  within-subjects  design  was  used  to 
control  for  potential  baseline  performance  differences  when  individuals  and  dyads  perform 
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the  same  task.  However,  as  deseribed  in  the  Results  seetion,  a  between-subjeets  faetor  was 
used  in  eomparing  data  from  individuals  and  dyads. 


Table  1,  List  of  participants  and  counterbalance  order.  Only  data  from  participants 
who  participated  in  both  the  individual  and  dyad  conditions  were  included  in  the 
analysis  (Italicized),  Dyads  consisted  of  individuals  with  an  equal  amount  of  experience 
with  the  task  and  the  same  counterbalance  order,  e.g,,  Dyad  6  consisted  of  two 
individuals  with  2  sessions  of  prior  experience  with  the  task,  once  as  an  individual  and 

once  as  part  of  a  dyad. 


PARTICIPANT 

SESSION  1 

SESSION  2 

SESSION  3 

A 

Individual  1 

No  dyad 

B 

Individual  2 

Dyad  3 

Dyad  6 

C 

Individual  3 

Dyad  4 

Dyad  6 

D 

Individual  4 

No  dyad 

E 

Individual  5 

No  dyad 

F 

Individual  6 

No  dyad 

G 

Individual  7 

Dyad  3 

Dyad  8 

H 

Individual  8 

No  dyad 

I 

Individual  9 

No  dyad 

J 

Individual  10 

Dyad  4 

Dyad  8 

K 

Callibration  problems  /  No  dyad 

L 

Callibration  problems  /  No  dyad 

M 

Dyad  2  /  Lost  data 

Dyad  11 

Individual  11 

N 

Dyad  5 

Dyad  11 

Individual  12 

0 

Dyad  2  /  Lost  data 

Dyad  9 

Individual  13 

P 

Dyad  7  /  No  Individual  from  partner 

Dyad  9 

Individual  14 

Q 

Dyad  5 

Individual  15 

R 

Dyad  1  /  No  Individual  from  partner 

Individual  16 

S 

Dyad  1  /  No  Individual  from  partner 

No  individual 

T 

Dyad  7 

No  individual 

Design 

The  design  of  this  study  is  based  direetly  on  Tollner  et  al.  (2006)  design  and  the  same 
task  was  used  in  both  studies.  However,  we  used  only  a  subset  of  the  levels  of  eaeh 
independent  variable  used  in  Tollner  et  al.  (2006)  study.  Two  between-subjeets  faetors  with 
two  levels  eaeh  (1)  Group:  Individual  ox  Dyad,  and  (2)  Communieation:  Communication  or 
No  communication  yielded  3  eonditions:  Individual,  Communication  {Comm)  dyads,  and  No 
communication  (No  comm)  dyads.  These  factors  were  not  completely  crossed,  as  the 
Communication  factor  applied  only  to  Dyads.  Within-subjects  factors  were  (1)  Set  size  with 
2  levels:  6  or  48  icons,  and  (2)  Change  magnitude  with  3  levels:  no,  small  or  large  change. 
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Stimuli  &  Apparatus 


Stimuli 

Stimuli  were  ereated  with  eustom  seripts  written  in  JAVA  version  1 .4.2.  Eaeh 
display  eonsisted  of  an  even  number  of  red,  blue,  and  green  squares  whieh  were 
approximately  1°  of  visual  angle  in  size  (i.e.,  squares  were  1.4  em  eaeh  viewed  from 
approximately  72  em)  and  randomly  distributed  aeross  the  entire  sereen.  Display  set  size 
was  6  or  48  squares  and  bloeked.  As  depieted  in  Figure  1,  a  trial  eonsisted  of  four  images 
presented  in  the  following  order  and  duration;  display  A  (750  ms),  mask  (250  ms),  either 
display  A,  or  A  with  one  square  ehanged  in  position  (750  ms),  mask  (250  ms).  The 
sequenee  repeated  until  the  partieipant  responded.  A  bloek  eonsisted  of  36  trials:  12  no 
change  and  24  change  trials  with  one  square  ehanged  in  position.  A  0.5°  visual  angle 
movement  in  any  direetion  was  a  small  change,  and  a  1.5°  visual  angle  movement  was  a 
large  change.  Change  magnitude  was  mixed  within  eaeh  bloek  and  randomly  presented.  A 
Pentium  4  desktop  eomputer  with  2  GB  of  RAM  was  used  to  present  the  displays  on  an  18” 
NEC  Multisyne  ECD  1800  flat  sereen  monitor.  In  the  Dyad  eondition,  two  independent 
eomputers  were  used  to  present  the  same  displays  on  two  separate  monitors. 
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(A) 


Figure  1,  Flicker  task  used  in  this  study.  The  top  panel  (A)  enotes  a  large  change,  set 
size  48  trial  and  the  bottom  panel  (B)  denotes  a  large  change,  set  size  6  trial. 


Eye  Tracker 

Two  ASL  model  501  head-mounted  eye  trackers  with  head  tracking  were  used  to  record 
eye  movements  (Figure  2).  This  system  measures  infrared  comeal  reflectance  and  pupil 
position  to  determine  point  of  gaze.  Fixations  were  points  of  gaze  with  maximum  change  of 
1°  visual  angle  for  a  minimum  of  100  ms.  Eye  movements  were  recorded  at  120  Hz  for  all 
but  four  participants  in  the  Individual  condition  whose  eye  movements  were  recorded  at  240 
Hz.  A  mixed  model  ANOVA,  with  sampling  rate  as  the  between-subjects  factor,  was  used  to 
compare  performance  and  number  of  fixations  from  these  two  groups  of  participants.  No 
significant  group  effects  or  interactions  emerged  (all  /?>.05),  therefore  data  from  the  two 
groups  were  combined  for  further  analyses. 
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Eye  tracker  error  resulted  in  artificial  fixations  and  fixations  which  fell  outside  of  the 
display  boundaries.  These  fixations  were  filtered  out  and  excluded  from  the  analyses. 
Artificial  fixations  were  fixations  that  were  preceded  by  saccadic  eye  movements  with  speeds 
greater  than  700°  visual  angle/sec.  The  eye  tracker  system  is  programmed  to  calculate  point 
of  gaze  using  head  position,  pupil  location,  and  corneal  reflection  (CR).  It  expects  the  CR 
created  by  the  infrared  light  source  on  the  head-mounted  camera  to  be  the  brightest  spot  on 
the  eye.  However,  in  some  cases,  reflections  from  other  light  sources  in  the  testing  room 
(e.g.,  monitors)  or  glossy  parts  of  the  eye  created  spots  that  were  at  least  as  bright  as  the  CR. 
This  caused  the  system  to  oscillate  between  tracking  the  actual  CR  and  the  brighter  spot, 
thereby  shifting  point  of  gaze.  Consequently,  artificial  fixations  were  created  whenever  point 
of  gaze  shifted  more  than  1°  of  visual  angle  for  a  period  exceeding  100  ms. 

Head  Tracker 

A  Flock  of  Birds  magnetic  tracker  was  used  to  track  head  movement.  Each  eye  tracker 
was  connected  to  its  own  control  unit,  with  the  two  control  units  configured  in  master/slave 
mode.  However,  as  shown  in  figure  2,  both  eye  trackers  shared  one  magnetic  transmitter 
which  was  positioned  between  the  two  stations,  approximately  3  feet  from  each  participant. 
The  Flock  of  Birds  magnetic  field  ranges  approximately  4  feet  and  samples  for  changes  in 
head  motion  at  approximately  144  Hz  (http://www.ascension- 

tech.com/products/flockofbirds.php).  An  eye/head  integration  mode  on  the  ASE  eye  tracker 
allowed  the  eye  tracker  to  subtract  head  movements  from  point  of  gaze  calculations. 

Heart  Rate  Monitor 

A  UFI  EZ-IBI  heart  rate  monitor  was  used  to  record  inter-beat  intervals  (IB I).  This 
system  required  three  surface  skin  electrodes  to  be  attached  to  the  participant  for  detection  of 
the  electrocardiogram  (ECG)  signal.  Two  electrodes  were  attached  to  the  chest:  one  on  the 
sternum  and  one  on  the  left  side  of  the  body  below  the  5*  rib,  approximately  6  inches  from 
the  armpit.  A  third  electrode  was  attached  to  the  right  wrist  and  served  as  the  ground.  The 
skin  below  the  electrodes  was  prepared  by  applying  NuPrep  ECG  and  EKG  skin  abrasion  gel 
to  the  site,  gently  wiping  the  skin  with  gauze,  then  cleansing  the  site  with  alcohol  prior  to 
application  of  the  electrodes.  The  UFI  EZ-IBI  monitor  recorded  data  at  the  rate  of  1000  Hz. 
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IBIs  were  time  periods  between  R  wave  peaks  and  were  filtered  off-line  using  Mierosoft 
Aeeess  version  1 1  from  Mierosoft  Offiee  Suite  2003. 

Workload  Questionnaires 

Partieipants  in  both  the  Individual  and  Dyad  eonditions  eompleted  an  eleetronic  version 
of  the  NASA  TLX  (Hart  &  Staveland,  1988)  which  is  comprised  of  6  subscales:  Mental 
demand,  Physical  demand,  Temporal  demand.  Effort,  Performance,  and  Frustration  (see 
Appendix  Cl).  Each  subscale  required  a  rating  between  0  and  100.  Ratings  from  the  6 
subscales  were  averaged  to  obtain  a  Raw  TLX  score  for  data  analysis  (RTLX:  Byers,  Bitner, 
&  Hill,  1989).  Dyad  RTLX  scores  were  obtained  by  averaging  subjective  workload  ratings 
from  both  members  (Bowers,  Braun,  &  Morgan,  1997).  Raw  data,  with  ratings  from  the 
individual  subscales,  were  also  analyzed  to  assess  differences  in  subscale  ratings  between 
groups. 

In  addition  to  the  NASA  TLX,  Dyads  also  completed  the  Team  Workload  Scale 
(Appendix  C2:  Hildebrand,  Pharmer,  &  Weaver,  2003).  This  questionnaire  consisted  of  five 
subscales:  Communication  demands.  Monitoring  demands.  Control  demands.  Coordination 
demands.  Leadership  demands,  with  3  questions  under  each  subscale.  Each  question 
required  a  rating  between  0  and  20  with  20  indicating  that  the  rater  experienced  high 
workload  under  a  specified  condition.  Dyad  subscale  scores  were  obtained  by  averaging 
ratings  from  members  of  each  dyad. 

Exit  Questionnaire 

All  participants  were  required  to  complete  an  exit  questionnaire  at  the  end  of  each  testing 
session  (Appendix  C3). 


Procedure 

Participants  were  seated  approximately  72  cm  from  the  screen.  In  the  Dyad  condition, 
members  viewed  the  same  displays  on  two  different  monitors.  As  shown  in  Figure  2  a 
cardboard  divider  was  used  to  prevent  members  of  a  dyad  from  viewing  each  other’s 
monitors.  Participants  were  asked  to  read  the  instructions  for  the  task  at  the  beginning  of 
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each  testing  session  (see  Appendix  B1  for  instructions  iox  Individuals  and  Appendix  2b  for 
Dyads) 

A  flicker  task  required  participants  to  indicate  by  key  press  if  one  square  in  an  array 
changed  in  position.  To  discourage  guessing,  participants  were  required  to  indicate  by 
mouse  click  which  square  changed  in  position  whenever  they  responded  that  a  change 
occurred.  All  participants  used  his  or  her  left  hand  on  the  keyboard  to  indicate  whether  or 
not  a  change  occurred  (“A”  for  YES  and  “D”  for  NO)  and  his  or  her  right  hand  on  the  mouse 
to  indicate  which  square  changed  in  position.  In  the  Dyad  condition,  participants  viewed  the 
same  displays  on  separate  monitors.  To  prevent  strategy  carry-over.  Dyads  were  tested  first 
in  the  No  comm  condition  in  which  communication  was  prohibited  before,  during,  and 
between  trial  blocks.  Dyads  were  then  tested  in  the  Comm  condition  in  which  they  were 
encouraged  to  communicate  and  develop  strategies  for  change  detection.  Participants 
completed  a  set  of  practice  trials  prior  to  each  condition.  Trials  were  self-paced  so  that  the 
subsequent  trial  began  only  when  the  participant  pressed  the  space  bar  to  continue. 

At  the  end  of  each  set  size  block,  participants  were  required  to  complete  a  NASA  TLX 
questionnaire  to  allow  us  to  determine  perceived  workload  and  validate  the  task  difficulty 
manipulation  of  set  size.  In  addition,  to  obtain  a  measure  subjective  perception  of 
performance  as  a  team,  participants  in  the  Dyad  condition  were  also  required  to  complete  the 
Team  Work  Load  Scale  at  the  end  of  each  set  size  block.  All  participants  completed  an  exit 
questionnaire  on  strategies  used  and  confidence  in  responses  given  during  the  task  at  the  end 
of  each  testing  session  (Appendix  C).  Participants  provided  confidence  ratings  on  a  scale 
from  1  to  10  with  10  indicating  the  highest  level  of  confidence  in  their  own  performance  of 
the  task. 
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(A)  (B) 

Figure  2,  (A)Two  ASL  head-  mounted  eye  trackers  were  used  to  record  eye 
movements,  (B)  A  divider  prevented  observers  from  viewing  each  others’  screens. 


Results 

Three  sets  of  analyses  were  performed  on  the  data.  The  first  set  below  is  comprised  of 
comparisons  of  data  between  groups,  i.e.,  Individuals,  No  comm  dyads,  and  Comm  dyads. 

The  second  set  focused  on  determining  the  efficacy  of  NNI  as  a  measure  of  workload.  The 
third  set  is  a  regression  model  predicting  change  detection  performance  in  dyads. 

Change  detection  performance  was  measured  using  response  time  (RT  in  ms)  and 
accuracy  (percentage  correct),  and  attention  allocation  and  scanning  strategy  were  examined 
by  analysis  of  fixations  and  scan  paths.  For  Dyads,  fixations  from  both  members  were 
summed  for  analysis  to  reflect  the  total  amount  of  effort  required  from  the  dyad.  However, 
each  member’s  percentage  of  re-scanned  trials  was  analyzed  separately  since  re-scanning 
varied  between  dyad  members  from  trial  to  trial,  e.g.,  one  member  re-scanned  on  trial  9,  but 
not  trial  10  and  the  other  member  re-scanned  on  both  trial  9  and  trial  10. 

Workload  was  assessed,  physiologically,  by  analyses  of  heart  rate  and  NNI,  and 
subjectively,  by  analyses  of  RTLX,  TLX  subscale  ratings,  and  Team  Workload  Scale  average 
ratings  by  set  size  block.  Average  heart  rate  and  NNI  were  obtained  by  combining  data 
across  change  magnitude  condition,  within  each  set  size  block.  For  dyads,  heart  rate  was 
obtained  by  averaging  both  dyad  members’  heart  rate  for  each  set  size.  Dyad  NNIs  were  also 
obtained  by  averaging  both  dyad  members’  NNI  for  each  set  size.  Confidence  ratings  on 
responses  given  during  the  task  were  averaged  between  dyad  members  and  compared  across 
groups  in  a  one-way  ANOVA. 
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Comparisons  between  groups 

Performance  data  along  with  number  of  fixations,  and  percent  of  re-scanned  and  cross- 
scanned  trials  were  submitted  to  mixed  model  ANOVAs.  To  determine  the  effect  of  having 
more  eyes  and  communication  on  change  detection,  two  separate  ANOVAs  were  performed 
to  compare  Individuals  to  No  comm  dyads  and  Individuals  to  Comm  dyads.  Data  were 
submitted  to  a  2  (group;  Individual/Dyad)  x  2  (set  size:  6/48  icons)  x  3  (change  magnitude: 
small/large/no  change)  mixed  model  repeated  measures  ANOVA  with  group  as  the  between- 
subjects  factor.  Although  all  participants  were  tested  in  both  Dyad  and  Individual  conditions, 
a  between-subjects  factor  was  used  since  responses  were  difficult  to  assign  to  a  specific 
member  in  a  dyad.  This  was  particularly  problematic  in  the  Comm  condition  in  which 
members  were  encouraged  to  collaborate  to  determine  a  response.  To  further  understand  the 
effect  of  communication  on  change  detection,  data  from  No  comm  and  Comm  dyads  were 
submitted  to  a  2  (group;  No  comm! Comm  dyad)  x  2  (set  size)  x  3  (change  magnitude) 
repeated  measures  ANOVA. 

To  determine  differences  in  workload  across  groups,  workload  measures,  i.e.,  RTLX, 

NNI  and  heart  rate  were  each  submitted  to  a  2  (set  size)  x  2  (group)  repeated  measures 
ANOVA  for  the  No  comm! Comm  dyad  analysis  and  a  2  (set  size)  x  2  (group)  mixed  model 
ANOVA  with  group  as  between-subjects  factor  for  the  Individual/No  comm  dyads  and 
Individual/Comm  dyads  analyses.  For  the  analysis  of  heart  rate,  no  significant  main  effects 
or  interactions  with  group  emerged.  Therefore,  heart  rate  data  from  all  groups  were 
combined  and  submitted  to  a  t-test.  Additionally,  to  determine  differences  in  workload 
between  groups  across  TLX  subscales,  another  set  of  ANOVAs  were  performed  on  the  TLX 
ratings.  NASA  TLX  ratings  were  submitted  to  a  2  (set  size)  x  6  (subscale)  repeated  measures 
ANOVA  for  the  No  comm/Comm  dyad  comparison  and  a  2  (group:  Individual/Dyad)  x  2  (set 
size)  X  6  (subscale)  mixed  model  ANOVA,  with  group  as  the  between  subjects  factor,  for  the 
Individual/No  comm  dyad  and  Individual/Comm  dyad  comparisons.  Team  Workload  Scale 
average  ratings  from  Comm  dyads  were  submitted  to  a  2  (set  size)  x  5  (subscale)  repeated 
measures  ANOVA.  Confidence  ratings  were  submitted  to  three  independent  unpaired  t-tests: 
No  comm/Comm,  Inidiv/No  comm,  Indiv/Comm.  A  Huynh-Feldt  (1970)  correction  factor  was 
used  to  adjust  for  violations  of  sphericity  on  all  repeated  measures  ANOVAs  and  a 
Bonferroni  correction  was  employed  for  all  post-hoc  comparisons  (unless  specified,  alpha 
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level  =  .017  sinee  most  tests  eondueted  required  three  pairwise  eomparisons).  Means  and 
standard  errors  for  RTs,  aeeuraey,  fixations,  and  pereent  of  re-seanned  and  eross-seanned 
trials  are  reported  in  Table  2. 

Praetiee  effeets  were  assessed  by  eomparison  of  RT  and  percent  correct  of  participants 
tested  as  individuals  first  and  last  in  a  mixed  model  ANOVA,  with  testing  order  as  the 
between-subjects  variable.  No  significant  group  effects  or  interactions  emerged  (all  j!7>.05). 

Response  Time 

No  Comm/Comm.  Overall,  RTs  were  slower  with  set  size  48  than  6,  F{\,6)  =  83.75, 
/><.05,  and  varied  with  change  magnitude,  F(2,12)  =  38.25, /><. 05.  All  post-hoc  comparisons 
were  significant  (p<.017).  RTs  were  slowest  to  no  change  and  fastest  to  large  change  trials. 
A  change  magnitude  x  set  size  interaction  emerged  from  the  analysis  of  RT,  F(2,12)  =  25.94, 
/><.05.  As  shown  in  Figure  3a,  the  change  magnitude  effect  was  driven  mostly  by  RT 
variations  with  change  magnitude  when  set  size  was  48. 

Individuals /No  comm.  Overall,  RTs  were  slower  with  set  size  48  than  6,  F(l,14)  =  73.59, 
/><.05,  and  varied  across  change  magnitude  F(2,28)  =  60.35, /><. 05.  All  post-hoc 
comparisons  were  significant  (p<.017),  with  RTs  slowest  to  no  change  and  fastest  to  large 
change  trials.  A  change  magnitude  x  set  size  interaction  emerged,  F(2,28)  =  37.58, /><. 05. 
Figure  3b  shows  that  the  change  magnitude  effect  was  due  mostly  to  RT  variations  with 
change  magnitude  when  set  size  was  48. 

Individuals/Comm.  Overall,  RTs  were  slower  with  set  size  48  than  6,  F(l,14)  =  66.92, 
/><.05,  and  Comm  dyads  were  faster  than  Individuals  to  detect  changes,  F(l,14)  =  4.98, 
/><.05.  A  set  size  x  group  interaction  emerged  F(l,14)  =  1 .56,p<.i)5.  As  shown  in  Figure  4, 
the  effect  of  group  on  RT  was  due  mostly  to  RT  differences  between  Individuals  and  Comm 
dyads  when  set  size  was  48,  such  that  Individuals  were  particularly  slower  than  Comm  dyads 
when  set  size  was  48.  Averaging  across  set  size  and  group,  RT  varied  across  change 
magnitude  F(2,28)  =  68.99, /><. 05.  All  post-hoc  comparisons  were  statistically  significant 
(p<.017),  with  RTs  slowest  to  no  change  and  fastest  to  large  change  trials.  A  change 
magnitude  x  set  size  interaction  emerged,  F(2,28)  =  45.56, /><. 05.  As  shown  in  Figure  3c, 
the  change  magnitude  effect  on  RT  was  driven  mostly  by  RT  variations  with  change 
magnitude  when  set  size  was  48. 
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A.  Comm  vs.  No  comm  dyads 


I 


A 


No  change  Small  change  Large  change 
Change  Magnitude 


B.  Individuals  vs.  No  comm 


No  change  Small  change  Large  change 

Change  Magnitude 


C.  Individuals  vs.  Comm  dyads 


No  change  Small  change  Large  change 
Change  Magnitude 


Figure  3,  RT  as  a  function  of  change  magnitude  for  set  size  6  and  48  (set  size  x  change 
magnitude  interaction,  p<,05).  Each  graph  represents  results  from  ANOVAs  with  the 
hetween-suhjects  factor  indicated  above  the  graph.  Striped  bars  represent  set  size  6  and 

solid  bars  represent  set  size  48, 
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Figure  4,  RT  as  a  function  of  set  size  for  Individuals  and  Comm  dyads  (set  size  x  group 
interaction,  p<,05).  Unfilled  bars  represent  Individuals  and  solid  bars  represent  Comm 

dyads. 
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Accuracy 

No  Comm/Comm.  Overall,  accuracy  was  lower  with  set  size  48  than  6,  F{\,6)  =  8.22, 
/><.05.  However,  a  group  x  set  size  interaction  emerged  from  the  analysis,  F{\,6)  =  21.30, 
p<.05.  Figure  5  shows  that  No  comm  dyads  were  less  accurate  than  Comm  dyads  particularly 
when  set  size  was  48.  Also,  overall,  accuracy  varied  with  change  magnitude,  F(2,12)  =  6.21, 
p<.Q5.  Post-hoc  comparisons  showed  lower  accuracy  on  small  compared  to  large  and  no 
change  trials, /><.0 17.  A  change  magnitude  x  set  size  interaction  emerged,  F(2,12)  =  6.65, 
/><.05.  Figure  6a  shows  that  the  effect  of  change  magnitude  on  accuracy  was  driven  mostly 
by  accuracy  variations  with  change  magnitude  when  set  size  was  48. 

Individuals/No  comm.  Overall,  accuracy  was  lower  with  set  size  48  than  6,  F{\,6)  = 
8.22,/»<.05,  and  varied  with  change  magnitude,  F(2,12)  =  \9.\3,p<.05.  Post-hoc 
comparisons  showed  lower  accuracy  on  small  compared  to  large  and  no  change  trials, 
p<.9\l.  A  set  size  x  change  magnitude  interaction  emerged,  F(2,12)  =  8.36,/><.05.  Figure 
6b  shows  that  the  change  magnitude  effect  on  accuracy  was  driven  by  accuracy  variations 
with  change  magnitude  when  set  size  was  48. 

Individuals/Comm.  Overall,  Individuals  were  less  accurate  than  Comm  dyads,  F(l,14)  = 
5.62,  j!7<.05.  On  average,  both  groups  were  less  accurate  with  set  size  48  than  6,  F(l,14)  = 
9.77,/><.05,  and  yielded  accuracy  variations  with  change  magnitude,  F(2,12)  =  17.45, 
j3<.05.  Post-hoc  comparisons  of  the  change  magnitude  effect  showed  lower  accuracy  on 
small  compared  to  large  and  no  change  trials, /><.0 17.  Figure  6c  depicts  a  set  size  change  x 
magnitude  interaction,  F(2,12)  =  5.54,/»<.05:  the  change  magnitude  effect  on  accuracy  was 
driven  by  accuracy  variations  with  change  magnitude  when  set  size  was  48.  This  analysis 
also  revealed  a  group  x  set  size  interaction  F(  1,1 4)  =  5.62,/»<.05,  and  a  group  x  change 
magnitude  interaction,  F(2,28)  =  3.52,/»<.05.  Figure  7  shows  that  the  effect  of  group  on 
accuracy  was  driven  by  accuracy  differences  when  set  size  was  48  than  when  set  size  was  6, 
and  as  shown  in  Figure  8,  by  greater  accuracy  variations  between  groups  on  Small  change 
trials. 
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B  No  comm  dyad  ■  Comm  dyad 


Figure  5,  Acuracy  as  a  function  of  set  size  for  No  comm  and  Comm  dyads  (set  size  x 
group  interaction,  p<,05).  Striped  bars  represent  No  comm  dyads  and  solid  bars 

represent  Comm  dyads. 


Figure  6,  Accuracy  as  a  function  of  change  magnitude  for  set  size  6  and  48  (set  size  x 
change  magnitude  interaction,  p<,05).  Each  graph  represents  results  from  ANOVAs 
with  the  between-subjects  factor  indicated  above  the  graph.  Striped  bars  represent  set 

size  6  and  solid  bars  represent  set  size  48, 
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Six  Forty-eight 

Set  size 


Figure  7,  Accuracy  as  a  function  of  set  size  for  Individuals  and  Comm  dyads  (group  x 
set  size  interaction,  p<,05).  Unfilled  bars  represent  Individuals  and  filled  bars 

represent  Comm  dyads. 
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Figure  8,  Accuracy  as  a  function  of  change  magnitude  for  Individuals  and  Comm 
dyads  (group  x  change  magnitude  interaction,  p<,05).  Unfilled  bars  represent 
Individuals  and  solid  bars  represent  Comm  dyads. 
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Table  2,  Means  and  standard  errors 
cross-scanned  trials  for  Individuals,  < 

SET  SIZE  6 

Large 

change 

RESPONSE  TIMEftn  ms) 
Individuals  1701.72 

No  Comm  teams  1 202.2 1 

Comm  teams  1173.50 

ACCURACY(%  correct) 
Individuals  100% 

No  Comm  teams  98.81% 

Comm  teams  98.81% 

NUMBER  OF  FIXATIONS 
Individuals  6.92 

No  Comm  teams  8.40 

Comm  teams  9.14 

%  RE-SCANNED  TRIALS 
Individuals  84.26% 

No  Comm  teams  63. 1 5% 

Comm  teams  71.75% 

%  CROSS-SCANNED  TRIALS 
No  Comm  teams  44.59% 

Comm  teams  32.68% 

SET  SIZE  48 

Large 

change 

RESPONSE  TIMEftn  ms) 

(B)  Individuals  7043.72 

No  Comm  teams  4684.43 

Comm  teams  3590.07 

ACCURACY(%  correct) 
Individuals  85.19% 

No  Comm  teams  88.10% 

Comm  teams  95.24% 

NUMBER  OF  FIXATIONS 
Individuals  18.44 

No  Comm  teams  21.93 

Comm  teams  20.65 

%  RE-SCANNED  TRIALS 
Individuals  97.22% 

No  Comm  teams  90.81% 

Comm  teams  90.69% 

%  CROSS-SCANNED  TRIALS 
No  Comm  teams  60.85% 

Comm  teams  1 1 .06% 


SE 

Small 

change 

SE 

No 

change 

SE 

383.63 

2639.83 

533.37 

4457.00 

650.90 

94.83 

1612.93 

165.90 

3015.71 

364.56 

58.38 

1535.86 

202.23 

3590.57 

490.39 

0.00 

85.19% 

0.06 

99.07% 

0.01 

0.01 

92.86% 

0.05 

98.81% 

0.01 

0.01 

95.24% 

0.02 

99.99% 

0.00 

0.95 

7.47 

1.92 

13.38 

2.55 

0.60 

9.77 

1.25 

15.40 

2.36 

0.32 

10.37 

0.69 

19.26 

1.46 

0.05 

84.72% 

0.05 

89.73% 

0.05 

0.04 

73.99% 

0.05 

87.28% 

0.05 

0.05 

77.27% 

0.05 

95.24% 

0.02 

0.06 

54.40% 

0.06 

81.82% 

0.06 

0.08 

39.63% 

0.08 

82.14% 

0.06 

Small 

No 

SE 

change 

SE 

change 

SE 

1576.92 

8996.83 

1512.61 

14942.06 

1996.52 

524.40 

3969.79 

429.75 

11054.93 

1503.09 

403.10 

3717.43 

427.15 

10015.07 

877.63 

0.08 

55.56% 

0.10 

100% 

0.00 

0.06 

77.38% 

0.08 

98.81% 

0.01 

0.02 

90.48% 

0.03 

100% 

0.00 

4.01 

11.38 

2.01 

39.41 

7.06 

2.22 

18.64 

2.20 

61.18 

9.77 

1.50 

22.83 

3.22 

54.55 

6.58 

0.02 

97.98% 

0.02 

100.00% 

0.00 

0.02 

95.73% 

0.02 

100.00% 

0.00 

0.03 

92.77% 

0.02 

99.40% 

0.01 

0.05 

53.59% 

0.08 

100.00% 

0.00 

0.03 

16.36% 

0.06 

71.43% 

0.10 
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Fixations 

No  comm/Comm.  Overall,  fewer  fixations  were  required  with  set  size  6  than  48,  F{\,6)  = 
57.87, /><.05.  However,  averaging  across  set  size  and  group  conditions,  the  number  of 
fixations  varied  across  change  magnitude,  F(2, 12)  =  28.07, /><. 05.  Post-hoc  comparisons 
showed  more  fixations  were  required  on  no  compared  to  small  and  large  change  trials, 
j!7<.017.  A  change  magnitude  x  set  size  interaction  emerged,  F(2,12)  =  22.82, /»<. 05.  Figure 
9a,  shows  that  the  effect  of  change  magnitude  on  fixations  was  driven  by  fixation  variations 
with  change  magnitude  when  set  size  was  48.  A  set  size  x  change  magnitude  x  group 
interaction  also  emerged,  F(2, 12)  =  1 .33,p<.05.  Figure  10  shows  that  more  fixations  were 
required  on  no  change  trials,  particularly  with  set  size  48,  with  a  greater  effect  on  No  comm 
than  Comm  dyads. 

Individuals /No  comm.  Overall,  fewer  fixations  were  required  with  set  size  6  than  48, 
F(l,14)  =  78.55, /><. 05.  A  set  size  x  group  interaction  emerged,  F(l,14)  =  4.66, ^><.05. 

Figure  1 1  shows  that  the  effect  of  set  size  on  number  of  fixations  was  due  mostly  to  No 
comm  dyads  than  Individuals.  Averaging  across  set  size  and  group,  the  number  of  fixations 
varied  with  change  magnitude  F(2,28)  =  49.87, /><.05.  Post-hoc  comparisons  showed  more 
fixations  on  no  compared  to  small  and  large  change  trials,  p<.0\l.  A  set  size  x  change 
magnitude  interaction  emerged,  F(2,28)  =  38.34, /><. 05.  Figure  9b  shows  that  the  change 
magnitude  effect  on  fixations  was  driven  by  fixation  variations  with  change  magnitude  when 
set  size  was  48.  Additionally,  a  set  size  x  change  magnitude  x  group  interaction  is  depicted 
in  Figure  12,  F(2,28)  =  3.54,  j!7<.05.  Both  groups  required  more  fixations  on  no  change  trials, 
particularly  with  set  size  48,  with  a  greater  effect  on  No  comm  dyads  than  Individuals. 

Individuals/Comm.  Overall,  fewer  fixations  were  required  with  set  size  6  than  48, 

F(l,14)  =  85.40,  p<.05.  However,  averaging  across  set  size  and  group,  the  number  of 
fixations  required  varied  with  change  magnitude,  F(2,28)  =  56.62, /»<. 05,  with  more  fixations 
on  no  compared  to  small  and  large  change  trials,  p<.i)\l.  A  set  size  x  change  magnitude 
interaction  emerged,  F(2,28)  =  28.31,/><.05.  Figure  9c  shows  that  the  change  magnitude 
effect  was  driven  by  change  magnitude  effects  on  number  of  fixations  when  set  size  was  48. 
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A.  Comm  vs.  No  comm 


B.  Individuals  vs.  No  comm 


C.  Individuals  vs.  Comm  dyads 


No  change  Small  change  Large  change  No  change  Small  change  Large  change  No  change  Small  change  Large  change 

Change  Magnitude  Change  Magnitude  Change  Magnitude 


Figure  9,  Number  of  fixations  as  a  function  of  change  magnitude  for  for  set  size  6  and 
48  (set  size  x  change  magnitude  interaction,  p<,05).  Each  graph  represents  results  from 
ANOVAs  with  the  between-subjects  factor  indicated  above  the  graph.  Striped  bars 
represent  set  size  6  and  solid  bars  represent  set  size  48, 
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Figure  10,  Number  of  fixations  as  a  function  of  change  magnitude  for  No  comm  and 
Comm  dyads  for  set  size  6  (on  the  left)  and  48  (set  size  x  change  magnitude  x  group 
interaction,  p<,05).  Striped  bars  represent  No  comm  dyads  and  solid  bars  represent 

Comm  dyads. 


23 


Figure  11,  Number  of  fixations  as  a  function  of  set  size  for  Individuals  and  No  comm 
dyads  (set  size  x  group  interaction,  p<,05).  Unfilled  bars  represent  Individuals  and 

striped  bars  represent  No  comm  dyads. 


Set  size  6 


Set  size  48 


Figure  12,  Number  of  fixations  as  a  function  of  change  magnitude  for  Individuals  and 
No  comm  dyads  for  set  size  6  (on  tbe  left)  and  set  size  48  (set  size  x  change  magnitude  x 
group  interaction,  p<,05).  Unfilled  bars  represent  Individuals  and  striped  bars 

represent  No  comm  dyads. 
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Cross-scanning 

No  Comm/Comm.  Overall,  No  comm  dyads  cross-scanned  more  than  Comm  dyads  F{\,6) 
=  10.13,/7<.05,  particularly  with  set  size  48,  F(\,6)  =  6.86,/><.05  (Figure  13;  Group  x  set 
size  interaction).  Figure  14  depicts  examples  of  typical  trials  for  AG  comm  and  Comm  dyads. 
Averaging  across  set  size  and  group,  percent  of  cross-scanned  trials  varied  with  change 
magnitude,  F(2,12)  =  43.99, /><. 05,  with  more  no  change  trials  cross-scanned  than  small,  or 
large  change  trials, /?<. 05.  A  change  magnitude  x  group  interaction  emerged,  ^(1,6)  =  5.71, 
/><.05.  Figure  15  shows  that  the  group  effect  on  percentage  of  cross-scanned  trials  was 
greatest  on  small  change  trials  and  least  on  no  change  trials. 
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Figure  13,  Percentage  of  cross-scanned  trials  as  a  function  of  set  size  for  No  comm  and 
Comm  dyads  (set  size  x  group  interaction,  p<,05).  Solid  bars  represent  Comm  dyads 
and  striped  bars  represent  No  comm  dyads. 
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Figure  14,  Scan  paths  for  Individuals,  No  comm  and  Comm  dyads  for  no  change  and 
change  trials.  The  two-toned  square  is  the  square  which  changed  in  position.  Solid  lines 
in  the  Dyad  trials  represent  the  scan  path  of  the  observer  that  responded. 
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Figure  15,  Percentage  of  cross-scanned  trials  as  a  function  of  change  magnitude  for  No 
comm  and  Comm  dyads  (change  magnitude  x  group  interaction,  p<,05).  Striped  bars 
represent  No  comm  dyads  and  solid  bars  represent  Comm  dyads. 
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Rescanning 

No  comm/Comm,  Overall,  more  trials  were  re-seanned  with  set  size  48  than  6,  F{\,6)  = 
35.18,/><.05.  However,  averaging  across  set  size  and  group,  percent  of  re-scanned  trials 
varied  across  change  magnitude,  F(2,12)  =  43.10,  p<.05.  Post-hoc  comparisons  showed  that 
more  no  change  trials  were  re-scanned  compared  to  small  and  large  change  trials,  /><.017.  A 
set  size  x  change  magnitude  interaction  emerged,  F(2,42)  =  18.35, /><. 05.  Figure  16a  shows 
that  the  change  magnitude  effect  on  percent  of  re-scanned  trials  was  driven  by  greater 
variation  in  re-scanning  when  set  size  was  48.  Additionally,  Figure  17  depicts  a  set  size  x 
group  interaction,  F(l,13)  =  6.82,  j!7<.05:  No  comm  dyads  re-scanned  more  trials  with  set 
size  48  than  6,  but  Comm  dyads  re-scanned  more  trials  when  set  size  was  6  than  48. 

Individuals /No  comm.  A  main  effect  of  group  was  only  marginally  significant.  On 
average.  Individuals  re-scanned  on  more  trials  than  No  comm  dyads,  F(l,21)  =  4.16, />  =  .05. 
Overall,  more  trials  were  re-scanned  with  set  size  48  than  6,  F(l,21)  =  32.46, /><. 05,  and  the 
percentage  of  re-scanned  trials  varied  across  change  magnitude,  F(2,42)  =  17.69, /><. 05. 
Post-hoc  comparisons  showed  that  more  no  change  trials  were  re-scanned  compared  to  small 
and  large  change  trials,  j!7<.0 17.  A  change  magnitude  x  set  size,  F(2,42)  =  6.43, /><.05  and 
change  magnitude  x  group  interaction  emerged,  F(2,42)  =  6.43,/><.05.  As  shown  in  Figure 
16b,  the  effect  of  change  magnitude  on  percent  of  re-scanned  trials  was  driven  by  variations 
in  percent  of  re-scanned  trials  with  change  magnitude  when  set  size  was  48,  and  also,  as 
shown  in  Figure  18,  by  greater  variations  in  re-scanning  across  change  magnitude  in  No 
comm  dyads  than  Individuals. 

Individuals/Comm.  On  average,  more  trials  were  re-scanned  when  set  size  was  48 
compared  to  6,  F(l,21)  =  25.88, /><. 05,  and  percent  of  re-scanned  trials  varied  by  change 
magnitude,  F(2,42)  =  14.03, /><. 05.  Post-hoc  comparisons  showed  that  more  no  change  trials 
were  re-scanned  than  small  or  large  change  trials,  j!7<.0 17.  Additionally,  a  change  magnitude 
X  group  interaction  emerged,  F(2,42)  =  4.78,/><.05.  Figure  19  shows  that  variations  in 
percentage  of  re-scanned  trials  across  change  magnitude  was  greater  in  Comm  dyads  than 
Individuals. 
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A.  Comm  vs.  No  comm 


B.  Individuals  vs.  No  comm 


Figure  16,  Percentage  of  re-scanned  trials  as  a  function  of  change  magnitude  for  set 
size  6  and  48  (set  size  x  change  magnitude  interaction,  p<,05).  Each  panel  represents 
results  from  ANOVAs  with  the  hetween-suhjects  factor  indicated  above  the  graphs. 
Striped  bars  represent  set  size  6  and  solid  bars  represent  set  size  48, 
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Figure  17,  Percentage  of  re-scanned  trials  as  a  function  of  set  size  for  No  comm  and 
Comm  dyads  (set  size  x  group  interaction,  p<,05).  Striped  bars  represent  No  comm 
dyads  and  solid  bars  represent  Comm  dyads. 
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Figure  18,  Percentage  of  re-scanned  trials  as  a  function  of  change  magnitude,  averaged 
across  set  size  (change  magnitude  x  group  interaction,  p<,05).  Unfilled  bars  represent 
Individuals  and  striped  bars  represent  No  comm  dyads. 
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Figure  19,  Percentage  of  re-scanned  trial  as  a  function  of  change  magnitude  for 
Individuals  and  Comm  dyads  (change  magnitude  x  group  interaction,  p<,05).  Unfilled 
bars  represent  Individuals  and  solid  bars  represent  Comm  dyads. 

Workload:  NASA  RTLX 

The  set  size  manipulation  hypothesized  to  drive  workload  was  validated  by  the  data.  As 
indicated  below,  all  ANOVAs  with  RTLX  yielded  significant  main  effects  of  set  size.  See 
Table  3  below  for  RTLX  means  and  standard  errors  for  all  groups. 

No  comm/Comm,  Overall,  No  comm  dyads  (M=  42.71,  SE  =  6.13)  had  higher  RTLX 
scores  than  Comm  dyads  (M=  35.54,  SE  =  5.78),  F(\,6)  =  17.15, /><. 05.  Additionally, 
averaging  across  group,  ratings  varied  by  set  size,  F{\,6)  =  13.56, /><. 05. 


29 


Individuals /No  comm.  Only  a  main  effect  of  set  size  emerged  from  this  analysis.  On 
average,  both  Individuals  and  No  comm  dyads  gave  higher  workload  ratings  when  set  size 
was  48  (M=  45.54,  SE  =  6.84)  compared  to  d  (M=  35.20,  SE  =  5.70),  F(l,14)  =10.23, ;7<.05. 

Individuals/Comm.  Only  a  main  effect  of  set  size  emerged  from  this  analysis.  Overall, 
both  Individuals  and  Comm  dyads  gave  higher  workload  ratings  when  set  size  was  48  (M  = 
42.82,  SE  =  6.80)  compared  to  6  (M=  30.74,  SE  =  5.38),  F(l,14)  =12.89, ^^<.05. 


Table  3.  Means  and  SEs  for  RTLX,  heart  rate,  and  NNI  for  Individuals,  No  comm  and 
Comm  dyads  and  Team  workload  scale  (TWS)  for  Comm  dyads. 


SET 

SIZE  6 

SE 

SET 

SIZE  48 

SE 

RTLX 

Individuals 

30.54 

2.03 

45.50 

5.30 

No  Comm  dyads 

39.86 

1.78 

45.57 

1.47 

Comm  dyads 

30.95 

1.51 

40.14 

1.85 

Heart  rate 

Individuals 

907.37 

67.88 

874.24 

49.49 

No  Comm  dyads 

862.12 

15.15 

844.49 

10.69 

Comm  dyads 

847.23 

11.14 

835.33 

9.31 

NNI 

Individuals 

0.86 

0.02 

0.87 

0.01 

No  Comm  dyads 

0.84 

0.02 

0.85 

0.01 

Comm  dyads 

0.85 

0.02 

0.86 

0.02 

TWS 

Comm  dyads 

3.40 

0.79 

3.87 

0.55 

Workload:  NASA  TLX  subscale  ratings 

No  comm/Comm,  Overall,  No  comm  dyads  (M=  42.71,  SE  =6.13)  gave  higher  ratings 
than  Comm  dyads  (M=  35.54,  SE  =  5.78),  F(\,6)  =  17.15,/><.05.  Additionally,  averaging 
across  group,  ratings  varied  by  set  size  with  higher  ratings  for  set  size  6  {M  =  35.40,  S.E.  = 
6.17)  than  set  size  48  6  (M  =  42.86,  S.E.  =  5.74),  F(l,6)  =  13.56, /><. 05,  and  by  subscale, 
F(l,6)  =  30.18,/><.05,  with  the  highest  ratings  for  performance  (M=  81.12,  S.E.  =  4.46), 
followed  by  mental  demand  (M=  40.68,  S.E.  =  8.01),  then  by  effort  (M=  37.66,  S.E.  =  6.71), 
then  by  temporal  demand  (M=  35.21 ,  S.E.  =  6.81),  then  by  frustration  (M=  22.98,  S.E.  = 
6.08),  and  last  by  physical  demand  (M=  17.07,  S.E.  =  3.67).  A  set  size  x  subscale  interaction 
is  depicted  in  Figure  20.  The  effect  of  subscale  on  ratings  appears  to  be  driven  by  greater 
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variability  in  ratings  across  subscale  for  set  size  48  eompared  to  set  size  6,  F(5,30)  =  3.94, 
p<.05.  Additionally,  a  group  x  subseale  interaetion  also  emerged,  F(5,30)  =  8.21,/»<.05.  As 
shown  in  Figure  21,  eompared  to  No  Comm  dyads,  Comm  dyad  gave  lower  ratings  on  all 
subseales  exeept  for  the  performance  seale,  in  whieh  they  rated  their  performanee  higher  than 
No  Comm  dyads. 

Individuals /No  comm.  Overall,  ratings  were  higher  set  size  was  48  {M=  45.54,  SE  = 

6.84)  eompared  to  6  (M=  35.20,  SE  =  5.70),  F(l,14)  =10.23, /><. 05.  Additionally,  ratings 
varied  by  subseale,  F(5,70)  =  32.06, /><. 05,  with  highest  ratings  for  performanee  {M=  76.39, 
S.E.  =  6.60),  followed  by  mental  demand  {M=  46.85,  S.E.  =  7.12),  then  by  effort  (M=  43.50, 
S.E.  =  6.59),  then  by  temporal  demand  (M=  37.38,  S.E.  =  7.17),  then  by  frustration  {M  = 
22.85,  S.E.  =  6.50),  and  last  by  physieal  demand  (M=  14.23,  S.E.  =  3.61).  A  set  size  x 
subseale  interaetion  is  depieted  in  Figure  22.  The  effeet  of  subseale  on  ratings  appears  to  be 
driven  by  greater  variability  in  ratings  aeross  subseale  for  set  size  48  eompared  to  set  size  6, 
F(5,70)  =  9.41,;7<.05. 

Individuals/Comm.  Overall,  both  Individuals  and  Comm  dyads  gave  higher  workload 
ratings  when  set  size  was  48  (M=  42.82,  SE  =  6.80)  eompared  to  6  {M=  30.74,  SE  =  5.38), 
F(l,14)  =12.89, /><. 05.  Additionally,  ratings  varied  by  subseale,  F(5,70)  =  40.92, /»<. 05, 
with  highest  ratings  for  performanee  {M=  79.12,  S.E.  =  6.77),  followed  by  mental  demand  (M 
=  40.60,  S.E.  =  7.56),  then  by  effort  (M=  37.44,  S.E.  =  7.01),  then  by  temporal  demand  (M  = 
30.22,  S.E.  =  7.22),  then  by  frustration  (M=  19.30,  S.E.  =  5.26),  and  last  by  physieal  demand 
{M=  14.01,  S.E.  =  3.34).  A  set  size  x  subscale  interaction  is  depicted  in  Figure  23.  The 
effect  of  subscale  on  ratings  appears  to  be  driven  by  greater  variability  in  ratings  aeross 
subseale  for  set  size  48  eompared  to  set  size  6,  F(5,70)  =  6.41, ^^<.05. 
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Figure  20,  TLX  ratings  as  a  function  of  subscale  for  set  size  6  and  48,  averaged  across 
Comm  and  No  comm  dyads  (set  size  x  subscale  interaction,  p<,05). 
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Figure  21,  TLX  ratings  as  a  function  of  subscale  for  Comm  and  No  comm  dyads, 
averaged  across  set  size  (group  x  subscale  interaction,  p<,05). 
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Figure  22,  TLX  rating  as  a  function  of  subscale  for  set  size  6  and  48,  collapsed  across 
Individuals  and  No  comm  dyads  (group  x  subscale  interaction,  p<.05). 
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Figure  23,  TLX  rating  as  a  function  of  subscale  for  set  size  6  and  48,  collapsed  across 
Individuals  and  Comm  dyads  (group  x  subscale  interaction,  p<,05). 
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Workload:  Heart  Rate 

ANOVAs  yielded  no  main  effects  or  interactions  with  group,  and  only  marginally 
significant  main  effects  of  set  size.  Therefore,  data  from  all  groups  were  combined  to 
increase  n  and  compared  in  a  paired  t-test  with  set  size  as  the  independent  variable  t(18)  = 
2.14,  p<.05.  Heart  rate  was  slower  when  set  size  was  6  {M=  868.54,  SE  =  24.61)  compared 
to  48  {M=  848.94,  SE  =  18.15),  suggesting  lower  workload,  overall,  with  set  size  6  than  48. 

Workload:  NNI 

No  comm/Comm,  Overall,  No  comm  dyads  {M=  .86,  SE  =  .01)  yielded  higher  NNIs,  thus 
more  random  fixation  patterns,  than  Comm  dyads  {M=  .82,  SE  =  .01),  F(l,13)  =  10.07, 
/><.05.  A  group  X  set  size  interaction  emerged,  F(l,13)  =  6.15,/i<.05.  As  shown  in  Figure 
24,  the  effect  of  group  on  NNI  was  driven  by  differences  when  set  size  was  48,  such  that  No 
comm  dyads  yielded  particularly  higher  NNIs  than  Comm  dyads  when  set  size  was  48. 

Individuals /No  comm.  No  significant  main  effects  or  interactions  emerged  from  this 
comparison. 

Individuals/Comm.  The  main  effect  of  group  and  the  interaction  of  group  with  set  size 
were  marginally  significant.  Individuals  had  higher  NNIs  (M=  .86,  SE  =  .01),  than  Comm 
dyads  (M=  .82,  SE  =  .01),  F(I,I4)  =  4.43, p=  .05.  A  group  x  set  size  interaction  was  also 
marginally  significant,  F(l,21)  =  4.23,  p  =  .06.  Figure  25  shows  that  Individuals  yielded 
particularly  higher  NNIs  than  Comm  dyads  when  set  size  was  48. 
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Figure  24,  NNI  as  a  function  of  set  size  6  and  48  for  No  comm  and  Comm  dyads  (group 
X  set  size  interaction,  p<,05).  Striped  bars  represent  No  comm  dyads  and  solid  bars 

represent  Comm  dyads. 
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Figure  25,  NNI  as  a  function  of  set  size  6  and  48  for  Individuals  and  Comm  dyads 
(group  X  set  size  interaction,  p<,05).  Unfilled  bars  represent  No  comm  dyads  and  solid 

bars  represent  Comm  dyads. 


Workload:  Team  Workload  Scale 

Overall,  average  team  workload  rating  varied  by  subscale,  F(4,24)  =  6.10,/><.05. 
Communication  demands  were  rated  highest  (M=  4.60,  SE  =  .75),  followed  by  Leadership 
demands  (M  =  4.12,  SE  =  .77),  Mental  demands  (M=  4.10,  SE  =  .71),  Coordination  demands 
(M=  3.20,  SE  =  .59),  with  Control  demands  rated  lowest  (M=  2.20,  SE  =  .53).  (A  set  size  x 
subscale  interaction  emerged,  F(4,24)  =  3.61  ,p<.05.  Figure  26  shows  that  variability  in 
average  rating  across  subscale  was  driven  by  greater  variability  when  set  size  was  48  than  6. 
Post-hoc  pairwise  comparisons  of  subscales  by  set  size  did  not  yield  any  significant 
differences  after  Bonferroni  adjustment  to  alpha  (alpha  level  .05/5  =  .01).  Only  the 
Communication  subscale  was  marginally  significant  before  Bonferroni  correction  was 
applied,  t{6)  =  -2.369,  p  =  .06.  As  shown  in  Figure  26,  average  rating  for  Communication 
demands  were  higher  when  set  size  was  48  {M=  5.43,  SE  =  .83)  compared  to  6  {M=  3.76,  SE 
=  .67). 

Confidence  Ratings 

An  average  confidence  rating  was  obtained  from  the  two  members  in  a  Dyad.  Only  7 
of  the  9  Individuals  completed  exit  questionnaires.  Therefore,  a  mean  score  was  obtained 
with  confidence  ratings  from  the  7  participants  in  the  Individual  condition  and  used  to  fill  in 
the  missing  scores  for  this  analysis.  A  paired  t-test  was  used  to  compare  Comm  and  No 
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comm  dyads,  and  unpaired  t-tests  were  used  to  compare  Individuals  to  Comm  dyads  and 
Individuals  to  No  comm  dyads.  Only  the  No  comm/Comm  dyad  comparison  yielded 
statistically  significant  results.  Comm  dyads  were  more  confident  in  their  responses  (M  = 
9.07,  S.E.  =  .35)  compared  to  No  comm  dyads,  t{6)  =  3.06,/><.05  (M=  8.43,  S.E.  =  .67).  The 
comparison  of  Individuals  {M=  8.29,  S.E.  =  1.09)  to  Comm  dyads  {M=  9.07,  S.E.  =  .35)  was 
only  marginally  significant,  t(14)  =  1.83,/>  =  .089  with  Comm  dyads  more  confident  in  their 
responses  than  Individuals. 
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Figure  26,  Average  dyad  rating  as  a  function  of  team  workload  demand  subscale  for  set 
size  6  and  48  (set  size  x  subscale  interaction,  p<,05).  Striped  bars  represent  set  size  6 

and  solid  bars  represent  set  size  48, 
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NNI  as  a  Workload  Measure 


To  perform  the  correlations  and  regressions  between  workload  measures,  data  from 
dyads  which  were  averaged  together  for  the  group  comparisons  were  again  separated  by  dyad 
member.  Dyad  members  were  treated  as  individual  participants. 

Evaluation  of  the  effectiveness  of  NNI  as  a  workload  measure  proceeded  in  the 
following  manner:  First,  RTLX  was  correlated  to  set  size  to  confirm  that  the  set  size 
manipulation  hypothesized  to  affect  workload  was  reflected  in  the  individual  RTLX  scores. 
Although  the  measure  was  validated  by  main  effects  of  set  size  from  the  ANOVAs 
performed  on  the  data,  this  analysis  was  performed  to  confirm  the  sensitivity  of  the  RTLX 
with  the  use  of  the  independent  dyad  member  RTLX  scores.  Second,  NNI,  heart  rate  and 
RTLX  were  correlated  following  confirmation  of  sensitivity  to  set  size  differences  of 
individual  dyad  member  RTLX  scores.  However,  only  a  subset  of  the  data  from  set  size  48 
were  included  in  the  correlations  of  workload  measures  due  to  the  following:  (1)  ceiling 
effects  on  NNIs  from  trials  with  set  size  6  (see  Figures  24,  25  and  section  below),  and  (2) 
heart  rate  data  were  incomplete  as  heart  rate  was  not  monitored  for  4  participants  in  the 
Individual  observer  condition  due  to  technical  difficulties  with  the  heart  rate  monitor. 

Ceiling  effect  and  confounding  factor  on  NNI  analysis 

NNI  is  the  ratio  of  the  average  distance  between  observed  and  randomly  distributed 
points  in  a  specified  area.  Our  NNI  data  from  the  set  size  6  condition  yielded  NNIs  close  to  1 
indicating  relatively  random  fixation  patterns.  More  importantly,  our  data  suggest  higher 
NNIs,  hence  higher  workload,  with  set  size  6  compared  to  set  size  48.  This  is  in  contrast  to 
previous  findings  by  Di  Nocera,  et  al.  (2006;  2007)  and  both  our  performance  and  NASA 
TLX  data  which  indicate  that  change  detection  was  more  difficult  with  set  size  48  than  with 
set  size  6. 

Due  to  inconsistency  with  both  performance  and  subjective  workload  data,  we 
determined  that  the  ceiling  effect  on  the  NNI  data  from  set  size  6  were  likely  driven  by  a 
confounding  factor  inherent  in  the  design  of  the  experiment  -  icons  were  randomly  positioned 
in  every  trial  to  counteract  potential  learning  effects.  As  a  result,  the  random  distribution  of 
icons  in  each  display  was  reflected  by  random  fixation  distributions  and  NNIs  close  to  1 . 
Although  icons  were  randomly  positioned  in  both  set  size  6  and  set  size  48,  some  dyads  in 
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the  Comm  condition,  in  addition  to  all  the  dyads  in  the  No  comm  condition,  searched  the 
entire  screen  independently  for  the  changing  icon  only  when  set  size  was  6.  In  contrast,  in 
the  set  size  48  condition,  all  dyads  in  the  Comm  condition  divided  the  screen  so  that  each 
member  searched  half  of  the  screen  for  a  change  in  icon  position.  In  some  cases,  one 
member  would  ask  the  other  to  confirm  that  none  of  the  icons  were  changing  in  position 
which  resulted  in  cross-scanning.  However,  as  reflected  in  the  scan  paths  of  Comm  dyads 
with  set  size  48  (Figure  14),  dyads  divided  the  search  space  in  half  on  most  trials.  This 
effectively  reduced  the  variability  in  distances  between  fixation  points  by  reducing  the  space 
in  which  more  points,  relative  to  set  size  6,  were  distributed.  Thereby,  on  average,  the 
likelihood  that  fixations  would  approach  a  random  distribution  or  an  NNI  value  close  to  1 
was  lower  when  set  size  was  48  compared  to  when  set  size  was  6. 

In  summary,  the  confounding  factor  described  above  was  due  to  the  difference  in 
search  space  across  task  difficulty  or  set  size  conditions.  While  icons  in  each  display  were 
randomly  distributed  across  both  set  sizes,  set  size  6  had  fewer  icons  and  a  wider  area  in 
which  icons  were  distributed,  and  thereby  greater  potential  variability  in  distances  between 
points  than  with  set  size  48  trials.  As  a  consequence,  the  likelihood  that  a  distribution  of 
fixations  would  be  close  to  a  random  distribution  was  greater,  on  average,  in  set  size  6  than 
set  size  48.  More  importantly,  compared  to  set  size  48,  the  displays  in  set  size  6  were  easier 
to  search  through  without  a  strategy  since  there  were  few  icons  within  the  field  of  view. 
Although  a  strategic  search  for  the  changing  icon  may  have  benefited  performance  with  both 
set  sizes,  a  random  search  was  effective  only  when  few  icons  were  on  the  display,  such  as  on 
set  size  6  trials. 
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RTLX and  set  size 

Results  indicate  a  positive  correlation  between  RTLX  and  set  size,  with  RTLX  higher 
for  the  larger  set  size,  set  size  48,  =  .12.  t  =  3.10,/><.05. 

NNI  and  RTLX 

NNI  and  RTLX  are  not  significantly  correlated. 

NNI,  heart  rate,  and  RTLX 

Correlations  between  NNI  and  heart  rate,  NNI  and  RTLX,  and  heart  rate  and  RTLX 
were  not  statistically  significant. 


Regression  model  of  change  detection  performance  with  multiple  observers 

Data  from  Comm  and  No  comm  dyads  were  submitted  to  a  backward  step-wise 
regression  analysis.  Our  goal  in  building  the  models  was  to  identify  the  best  predictors  of 
efficient  change  detection  with  multiple  observers.  Such  a  model  could  inform  both  the 
development  of  training  methods  for  change  detection  and  adaptive  technologies  to  support 
team  performance. 

RT  and  accuracy  were  used  as  dependent  variables  in  2  independent  models  of 
change  detection  performance.  Predictors  were  number  fixations,  percentage  of  re-scanned 
and  cross-scanned  trials,  heart  rate,  and  RTLX.  NNI  was  not  included  in  the  analysis  due  to 
the  confounding  factor  on  data  from  set  size  6,  as  discussed  above. 

Response  time  model 

Number  of  fixations  and  RTLX  were  the  only  significant  predictors  of  RT,  F(2,25)  = 
63.64, /»<. 05,  see  Tables  3  for  the  ANOVA  and  regression  tables.  The  regression  equation  y' 
=  -  660.04  +  162.441  Fixations  +  27.421RTLX  predicts  82.3%  of  the  variance  in  RT  in 
change  detection  with  multiple  observers,  adjusted  R  =  .823.  See  Table  3a  and  3b  for  the 
regression  and  ANOVA  tables. 
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Table  3.  Regression  and  ANOVA  tables  for  the  model  of  RT  in  change  detection  with 

multiple  observers. 


Effect 

Coefficient 

Std  Error 

Std  Coef 

Tolerance 

t 

P(2  Tail) 

CONSTANT 

-660.04 

689.33 

0 

-0.958 

0.347 

FIXATIONS 

1 62.44 1 

15.133 

0.881 

0.975 

10.734 

0.000 

RTLX 

27.421 

15.96 

0.I4I 

0.975 

1.718 

0.098 

Analysis  of  Variance 

Source 

Sum-of-Squares 

df 

Mean-Square 

F-ratio 

P 

Regression 

I.27E-r08 

2 

6.35E+07 

63.638 

0.000 

Residual 

2.49E-r07 

25 

997304.883 

Accuracy  model 

All  but  one  entered  into  the  model  were  signifieant  predietors  of  aeouracy,  F{5,22)  = 
15.21, /»<. 05,  see  Tables  4  for  the  ANOVA  and  regression  tables.  RTLX  reached  marginal 
significance  atp  =  .067.  The  regression  equation  y'  =  2.192  +  .002Fixations  -.095Cross-scan 
%  -.603Re-scan  %  +-.001RTLX  +  -.OOlheart  rate  and  predicts  72.5%  of  the  variance  in 
accuracy  of  change  detection  with  multiple  observers,  adjusted  R^=  .732.  See  Table  4  for  the 
regression  and  ANOVA  tables. 


Table  4,  Regression  and  ANOVA  tables  for  the  model  of  accuracy  in  change  detection 

with  multiple  observers. 


Effect 

Coefficient 

Std  Error 

Std  Coef 

Tolerance 

t 

P(2  Tail) 

CONSTANT 

2.192 

0.175 

0.000 

12.498 

0.000 

FIXATIONS 

0.002 

0.001 

0.409 

0.381 

2.500 

0.020 

CROSS-SCAN 

-0.095 

0.039 

-0.254 

0.957 

-2.465 

0.022 

RE-SCAN 

-0.603 

0.129 

-0.855 

0.307 

-4.690 

0.000 

RTLX 

-0.001 

0.001 

-0.226 

0.738 

-1.924 

0.067 

HEART  RATE 

-0.001 

0.000 

-0.796 

0.548 

-5.838 

0.000 

Analysis  of  Variance 

Source 

Sum-of-Squares 

df 

Mean-Square 

F-ratio 

P 

Regression 

0.127 

5.000 

0.025 

15.207 

0.000 

Residual 

0.037 

22.000 

0.002 

Discussion 


The  advantage  provided  by  having  more  eyes/resourees  for  ehange  deteetion  depends  on 
whether  or  not  (1)  observers  are  allowed  to  eommunieate,  and  (2)  a  change  occurs.  Our 
results  provide  direct  evidence  that  communication  allows  efficient  attention  allocation  and 
scanning  strategies  for  change  detection  by  allowing  observers  to  function  as  a  team.  The 
results  of  the  regression  analyses  predicting  performance  solidify  the  importance  of  attention 
allocation  and  scanning  strategies  in  change  detection  with  multiple  observers. 

Overall,  all  workload  measures  used  in  the  present  study,  including  NNI,  were  sensitive 
to  changes  in  task  difficulty.  Additionally,  RTLX  and  NNI  both  reflected  differences  in 
workload  between  communicating  and  non-communicating  dyads,  with  both  measures 
indicating  higher  workload  in  non-communicating  compared  to  communicating  dyads. 
However,  none  of  the  workload  measures  were  statistically  correlated  to  one  another  and 
results  of  the  NNI  analysis  revealed  some  limitations  for  using  NNI  to  index  workload. 


Are  effects  of  communication  on  change  detection  performance  apparent  in  eye  movements 

and  scan  paths? 

Performance  results  are  consistent  with  those  from  Tollner  et  al.  (2006).  Overall, 
communicating  dyads  detected  changes  faster  and  more  accurately  than  individuals. 
Likewise,  communicating  dyads  outperformed  both  non-communicating  dyads  and 
individual  observers  when  the  task  was  difficult.  Our  goal  was  to  determine  if  the  results  of 
Tollner  et  al.  (2006),  replicated  in  the  present  study,  may  be  attributed  to  attention  allocation 
and  scanning  strategies  afforded  not  just  by  availability  of  resources,  but  by  the  ability  of 
multiple  observers  to  communicate  and  function  as  a  team. 

First,  our  data  show  that  communicating  dyads  functioned  as  a  team,  with  several  major 
components  of  teamwork  evident  in  their  scan  paths:  back-up  behavior,  coordination, 
feedback,  and,  of  course,  communication  (Dickinson  &  McIntyre,  1997).  Although  cross¬ 
scanning  was  evident  in  both  communicating  and  non-communicating  dyads,  cross-scanning 
in  communicating  dyads  was  less  frequent  and  deliberate  for  confirmation  of  change  or  no 
change.  Additionally,  designated  areas  of  responsibility  are  evident  in  the  scan  paths  of 
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communicating  dyads  only  (see  Figure  14).  In  eontrast,  the  inferior  performance  of 
individual  observers  and  non-communicating  dyads  may  be  due  to  the  fact  that  they  served  as 
their  own  feedback  mechanism,  and  lacked  the  ability  to  coordinate  their  search  strategies  for 
optimal  performance. 

Seeond,  analyses  of  both  fixations  and  sean  paths  support  the  idea  that  individuals  and 
teams  utilize  different  attention  allocation  and  scanning  strategies  in  change  detection.  Non¬ 
communicating  dyads  (1)  required  more  fixations,  thereby  more  re-alloeation  of  attention, 
than  communicating  dyads  and  individuals  to  detect  changes,  and  (2)  cross-scanned  more 
trials  than  communicating  dyads,  particularly  when  the  task  was  diffieult  (set  size  48). 
However,  non-eommunieating  dyads  gained  no  obvious  performanee  benefits  over 
individuals  and  communicating  dyads.  Instead,  they  were  slower  than  individuals  and  less 
aecurate  than  eommunieating  dyads  when  the  task  was  difficult.  Overall,  non¬ 
communicating  dyads  used  more  time-consuming,  less  efficient  strategies  than  both 
individuals  and  eommunieating  dyads.  In  addition,  these  findings  suggest  that  the  benefits  of 
multiple  observers  in  change  detection  are  undermined  by  the  lack  of  a  medium  sueh  as 
eommunication  for  coordinating  actions  and  determining  strategies. 

Results  from  our  eye  movement  analyses  are  eonsistent  with  Zelinsky’s  (2001)  findings 
that  eye  movements  in  change  detection  reflect  information  gathering  and  change 
confirmation.  This  is  evident  in  the  use  of  re-scanning  by  all  three  groups  and  eross-scanning 
in  eommunieating  dyads.  More  importantly,  results  showed  that  communication  allowed 
dyads  to  develop  strategies  for  effieient  resource  allocation  in  the  service  of  information 
gathering  and  change  confirmation.  While  cross-scanning  appeared  to  be  a  costly,  tacit 
behavior  for  non-communicating  dyads,  it  served  as  an  effeetive  strategy  for  eonfirmation  of 
change  for  communicating  dyads. 

Interestingly,  while  eommunieating  dyads  were  better  than  individuals  at  deteeting 
ehanges,  trials  that  laeked  change  were  costly  for  both  individuals  and  dyads  regardless  of 
communication  condition.  On  no  change  trials,  both  individual  observers  and  dyads  were 
slower  to  respond  and  required  more  fixations.  This  finding  has  implieations  for 
environments  sueh  as  tactical  C2  in  which  a  common  and  current  understanding  of  a  visual 
environment  is  neeessary  for  decision-making.  Delayed  information  updating  due  to  slow 
confirmation  of  the  lack  of  change  by  one  or  more  members  of  a  distributed  team  could  delay 
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decision-making  and  potentially  lead  to  catastrophic  outcomes.  Likewise,  cognitive 
resources  consumed  by  slow  confirmation  of  no  change  may  be  better  allocated  to  other 
important  tasks. 

Finally,  Tollner  et  al.  (2006)  speculated  that  the  slowed  responses  on  trials  in  which  no 
change  occurred  may  have  resulted  from  a  serial  and  exhaustive  scanning  of  the  entire 
display,  as  previously  seen  in  visual  search  studies  (e.g.,  Treisman  &  Gelade,  1980).  Our 
fixation  data  provide  some  support  for  this  speculation.  On  trials  with  no  change,  on 
average,  individuals  required  approximately  13  fixations  with  set  size  6  and  40  fixations  with 
set  size  48.  Likewise,  communicating  dyads  required  approximately,  19  fixations  with  set 
size  6  and  54  with  set  size  48,  non-communicating  dyads  required  approximately  15  fixations 
with  set  size  6  and  61  with  set  size  48.  These  fixation  numbers  are  approximately  twice 
those  from  trials  in  which  a  change  occurred  and  are  almost  equal  to  the  number  of  stimuli  in 
the  display.  Moreover,  visual  inspection  of  scan  paths  suggest  that  more  of  the  display  was 
searched  on  a  majority  of  trials  when  no  change  occurred  compared  to  when  a  change  did 
occur  (see  Figure  14  and  Figure  27).  However,  a  more  thorough  analysis  of  scan  paths  is 
necessary  to  confirm  this  hypothesis.  Figure  27  below  are  representative  samples  of  scan 
paths  from  individual  observers,  communicating  and  non-communicating  dyads. 

Findings  on  scanning  strategies  may  be  used  for  training  and  development  of  adaptive 
aids  and  technologies  to  support  team  performance.  First,  evidence  suggests  that  scanning 
strategies  used  in  search  can  be  trained  (Wang,  Lin,  &  Drury,  1997).  Therefore, 
identification  of  efficient  scanning  patterns  may  be  used  in  training  both  individual  operators 
and  teams  to  search  for  relevant  changes  more  effectively.  Second,  identification  of 
inefficient  scanning  patterns  may  be  applied  to  the  development  of  adaptive  aids  for 
individuals  and  teams  showing  signs  of  inefficient  change  detection  or  search  behavior. 
Finally,  an  understanding  of  when  change  detection  strategies  could  delay  decision-making 
can  guide  the  design  of  interfaces  used  by  highly  synchronized,  distributed  teams,  such  as 
teams  in  tactical  C2.  The  importance  of  understanding  the  patterns  of  eye  movements  for 
both  training  and  development  of  adaptive  aids  is  supported  by  our  models  which  revealed 
that  number  of  fixations,  re-scanning,  and  cross-scanning  are  significant  predictors  of  either 
change  detection  speed,  or  change  detection  accuracy,  or  both. 
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Set  size  6 


Cofnm  dyads 


Set  size  48 


(C) 


Figure  27,  (A)  Scan  paths  for  Individuals  and  No  comm  dyads  for  trials  with  No 
change,  Small  and  Large  change.  Scan  paths  of  Individuals  are  shown  against  a  gray 
background,  (B)  A  greater  portion  of  No  change  displays  were  scanned  compared  to 
Small  and  Large  change  displays,  (C)  Scan  paths  for  Comm  dyads  for  trials  with  No 

change,  Small,  and  Large  change. 


Is  the  NNI  a  viable  workload  index? 

Our  data  suggest  that  the  NNI  is  sensitive  to  variations  in  task  diffieulty  and  subjeetive 
workload,  but  with  eertain  constraints.  First,  given  that  the  NNI  is  the  ratio  of  the  average 
distance  between  actual  and  random  points  within  a  space,  it  may  be  a  valid  workload  metric 
only  when  the  comparison  is  between  conditions  with  the  same  relevant  visual  space.  More 
importantly,  this  is  regardless  of  whether  or  not  the  task  within  the  space  is  static  or  dynamic 
in  nature.  The  metric  may  fall  short  as  a  workload  index  in  experiments  such  as  the  present 
in  which  the  size  of  the  relevant  visual  space  is  uncontrolled  and  may  vary  both  by  condition 
and  by  participant.  Second,  the  reliability  of  the  metric  itself  depends  on  the  number  of 
points  used  in  the  calculation  with  more  points  yielding  more  reliable  the  NNI.  Therefore, 
the  measure  may  be  unreliable  when  used  to  measure  workload  in  tasks  that  require  or  allow 


45 


few  fixations,  regardless  of  diffieulty.  For  example,  in  air  battle  management,  situations  may 
vary  within  relatively  short  periods  of  time,  thereby  allowing  only  few  fixations  within  eaeh 
relevant  time  period.  Lastly,  the  laek  of  signifieanee  in  the  eorrelation  between  NNI  and 
other  workload  measures  indieates  that  more  work  is  neeessary  to  validate  the  NNI  as  a 
reliable  index  of  eognitive  workload. 
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Appendix  A:  Consent  Form  and  Biographical  Questionnaire 
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INFORMATION  PROTECTED  BY  THE  PRIVACY  ACT  OP  1974 


Informed  Consent  Document 
For 

Individual  and  Team  Susceptibility  to  Change  Blindness 

AFRE/HECP,  WPAFB,  OH,  Building  33 


Princinal  Investigator; 

W.  Todd  Nelson,  DSN-785-8803,  AFRE/HECP, 

william.nelson(a),wDafb.afmil 

Associate  Investigators: 

Rebecca  D.  Brown,  DSN-785-0884,  AFRE/HECP, 

beckv.brown(alwpafb .  af  mil 

1 .  Nature  and  purpose:  You  have  been  offered  the  opportunity  to  participate  in  the 
“Individual  and  Team  Susceptibility  to  Change  Blindness”  research  study.  Your 
participation  will  occur  sometime  between  01  November  2006  and  01  November  2007,  in 
the  BCM2  Lab  at  Wright-Patterson  AFB. 
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The  purpose  of  this  research  is  to  identify  conditions  under  which  individual  operators 
and  teams  may  be  vulnerable  to  change  blindness,  the  striking  phenomenon  in  which 
observers  often  fail  to  detect  changes  to  objects  in  visual  scenes  from  one  view  to  the 
next.  This  study  also  aims  to  determine  whether  teams  are  less  susceptible  to  change 
blindness  than  individual  operators,  and  to  identify  strategies  used  by  teams  to  detect 
changes. 

The  time  requirement  for  each  volunteer  participant  is  anticipated  to  be  a  total  of  1  visit 
of  approximately  2  hours.  Approximately  150  participants  will  be  enrolled  in  this  study. 
You  will  be  required  to  have  normal  or  corrected-to-normal  visual  acuity  to  be  eligible 
for  participation. 

2.  Experimental  procedures:  If  you  decide  to  participate  you  will  be  asked  to  view 
tactical  displays  on  a  desktop  PC  with  the  task  of  detecting  heading,  color,  and  position 
changes  either  individually  or  as  part  of  a  team.  Testing  will  last  approximately  2  hours 
and  will  be  done  in  normal  lighting  conditions.  Normally,  you  will  be  seated  during  the 
data  collection  trials.  You  will  be  offered  a  rest  period  midway  through  the  testing  period 
but  can  request  a  break  at  any  time  during  the  experiment.  You  may  withdraw  this 
consent  at  any  time  and  discontinue  further  participation  in  this  study  without  prejudice 
to  your  entitlements.  Also  understand  that  the  medical  monitor  of  this  study  may 
terminate  your  participation  in  this  study  if  she  or  he  feels  this  to  be  in  your  best  interest. 
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While  performing  the  simulation  behavioral  and  physiological  measures  will  be  collected. 
Specifically,  either  eye  movements,  or  heart  rate,  or  both  will  be  recorded.  A  n  A  SL 
Model  501  head-mounted  eye  tracker  will  be  used  to  record  your  eye 
movements  and  an  electrocardiogram  (ECG)will  be  used  to  record  your  heart 
rate.  None  of  these  methods  are  physically  invasive. 

3.  Discomfort  and  risks:  There  is  minimal  risk  and/or  discomfort  associated  with 
performing  this  task.  Mild  postural  fatigue  and  eye  strain  has  been  shown  to  be  prevalent 
in  continued  computer  usage  but  is  normally  alleviated  with  rest  breaks.  There  is  a  small 
risk  that  the  leads  used  for  the  heart  tracing  (ECG)  may  irritate  your  skin. 

4.  Precautions  for  female  participants:  There  are  no  special  precautions  for  female 
participants. 

5.  Benefits:  You  are  not  expected  to  benefit  directly  from  participation  in  this  research 
study. 

6.  Compensation:  You  will  receive  the  standard  compensation  of  $15.00  per  hour  (cash)  as 
a  member  of  the  General  Dynamics  paid  volunteer  subject  pool. 

7.  Alternatives:  Choosing  not  to  participate  is  an  alternative  to  volunteering  for  this  study. 

8.  Entitlements  and  confidentiality: 
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a.  Records  of  your  participation  in  this  study  may  only  be  disclosed  according  to  federal 


law,  including  the  Federal  Privacy  Act,  5  U.S.C.  552a,  and  its  implementing 
regulations. 

b.  You  understand  your  entitlements  to  medical  and  dental  care  and/or  compensation  in 
the  event  of  injury  are  governed  by  federal  laws  and  regulations,  and  that  if  you  desire 
further  information  you  may  contact  the  base  legal  office  (88  ABW/JA,  257-6142  for 
Wright-Patterson  AFB).  You  may  contact  the  medical  monitor.  Dr.  Jeff  Bidinger, 
Maj.,  USAF,  of  this  research  study  at  (937)  656-5449. 

c.  If  an  unanticipated  event  (medical  misadventure)  occurs  during  your  participation  in 
this  study,  you  will  be  informed.  If  you  are  not  competent  at  the  time  to  understand 
the  nature  of  the  event,  such  information  will  be  brought  to  the  attention  of  your  next 
of  kin. 


Next  of  Kin  if  needed.  Name 
Phone# 


d.  The  decision  to  participate  in  this  research  is  completely  voluntary  on  your  part.  No 
one  has  coerced  or  intimidated  you  into  participating  in  this  program.  You  are 
participating  because  you  want  to.  Todd  Nelson,  or  an  associate,  has  adequately 
answered  any  and  all  questions  you  have  about  this  study,  your  participation,  and  the 
procedures  involved.  Todd  Nelson  can  be  reached  at  (937)  255-8803.  You 
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understand  that  Alison  Tollner,  or  an  associate  will  be  available  to  answer  any 
questions  concerning  procedures  throughout  this  study.  You  understand  that  if 
significant  new  findings  develop  during  the  course  of  this  research,  which  may  relate 
to  your  decision  to  continue  participation,  you  will  be  informed.  You  further 
understand  that  you  may  withdraw  this  consent  at  any  time  and  discontinue  further 
participation  in  this  study  without  prejudice  to  your  entitlements.  You  also 
understand  that  the  medical  monitor  of  this  study  may  terminate  your  participation  in 
this  study  if  she  or  he  feels  this  to  be  in  your  best  interest.  If  you  have  any  questions 
or  concerns  about  your  participation  in  this  study  or  your  rights  as  a  research 
participant,  please  contact  Major  Jeff  Bidinger  at  (937)  656-5449  or 
i  effrev.bidinger@wpafb.  af  mil. 

e.  You  understand  that  your  participation  in  this  study  may  be  photographed,  filmed  or 
audio/videotaped.  The  audio/video  data  will  be  used  for  data  analysis,  data  retrieval 
and  backup  purposes  only.  All  audio/video  media  will  be  stored  in  a  secure  cabinet 
for  up  to  5  years  in  the  BMC2  Lab,  Bldg  33,  WPAFB,  OH.  You  consent  to  the  use  of 
these  media  for  training  and  data  collection  purposes  and  understand  that  any  release 
of  records  of  your  participation  in  this  study  may  only  be  disclosed  according  to 
federal  law,  including  the  Federal  Privacy  Act,  55  U.S.C.  552a,  and  its  implementing 
regulations.  This  means  personal  information  will  not  be  released  to  unauthorized 
source  without  your  permission. 
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f.  YOU  FULLY  UNDERSTAND  THAT  YOU  ARE  MAKING  A  DECISION 


WHETHER  OR  NOT  TO  PARTICIPATE.  YOUR  SIGNATURE  INDICATES 
THAT  YOU  HAVE  DECIDED  TO  PARTICIPATE  HAVING  READ  THE 
INEORMATION  PROVIDED  ABOVE. 


Volunteer 

Signature _ Date 


Volunteer  Social  Security  No.  (Optional) 


Advising  Investigator  Signature 


Date 


Witness  Signature 


Date 


Privacy  Act  Statement 

Authority:  We  are  requesting  diselosure  of  personal  information,  to  inelude  your  Soeial 
Seeurity  Number.  Researehers  are  authorized  to  eolleet  personal  information  (ineluding 
soeial  seeurity  numbers)  on  researeh  partieipants  under  The  Privaey  Aet-5  USC  552a,  10 
use  55,  10  USC  8013,  32  CER  219,  45  CER  Part  46,  and  EO  9397,  November  1943. 
Purpose:  It  is  possible  that  latent  risks  or  injuries  inherent  in  this  experiment  will  not  be 
diseovered  until  some  time  in  the  future.  The  purpose  of  eolleeting  this  information  is  to  aid 
researehers  in  loeating  you  at  a  future  date  if  further  diselosures  are  appropriate. 

Routine  Uses:  Information  (ineluding  name  and  SSN)  may  be  furnished  to  Eederal,  State 
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and  local  agencies  for  any  uses  published  by  the  Air  Foree  in  the  Federal  Register,  52  FR 
16431,  to  inelude,  furtheranee  of  the  research  involved  with  this  study  and  to  provide 
medieal  eare. 

Disclosure:  Diselosure  of  the  requested  information  is  voluntary.  No  adverse  aetion 
whatsoever  will  be  taken  against  you,  and  no  privilege  will  be  denied  you  based  on  the  faet 
you  do  not  disclose  this  information.  However,  your  participation  in  this  study  may  be 
impaeted  by  a  refusal  to  provide  this  information. 
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Individual  and  Team  Susceptibility  to  Change  Blindness  -  Eye  Tracking 


Study 

Participant  Biographical  Form 


Participant  Name: _  Date: _ 

Age: _ 

Gender:  Male  Female 

Handedness:  Left  Right  Both 

Vision:  Normal  20/20  Correeted  to  Normal  Defieient 

Education  level: 
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Total  years  of  military  experience: 


Rank: 


How  many  hours  per  week  do  you  play  video/computer  games? 


1.)  0  hrs/week  2.)  1-10  hrs/week  3.)  11-20  hrs/week 


4,)  21-30  hrs/week  5.)  30+  hrs/week 


Using  the  following  scale,  indicate  how  much  you  enjoy  playing  video/computer  games? 


Not  at  All 


Enjoy 


Very  much 


Enjoy 
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Appendix  B:  Task  Instructions 
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Appendix  B 1 :  Instructions  for  Individual  Observers 
Instructions  for  change  detection  task 

You  will  see  an  image  flashing  on  the  screen  with  either  6  or  48  colored  squares.  On 
some  trials,  one  of  the  squares  will  ehange  in  position.  Your  job  is  to  indicate  (1)  if  one  of 
the  squares  ehanges  in  position  during  a  trial  and  (2)  whieh  square  changed  in  position. 

Please  press  “YES”  to  indieate  that  you  saw  one  of  the  squares  change  in  position  and  “NO” 
if  you  think  that  none  of  the  squares  changed  in  position.  On  trials  in  which  you  respond 
“YES,”  that  a  ehange  in  position  oeeurred,  you  will  be  asked  to  indicate  whieh  square 
ehanged  in  position  by  using  the  mouse  to  select  it.  Please  respond  as  quickly  AND 
accurately  as  you  can.  Press  the  space  bar  to  move  on  to  the  next  trial. 

The  task  consists  of  2  sets  of  trials,  one  set  with  only  6  squares  on  the  sereen  and  one 
set  with  48  squares  on  the  sereen.  The  number  of  squares  on  the  screen  will  not  vary  within  a 
set  of  trials.  You  will  start  with  either  one  or  the  other  set  and  you  will  be  given  a  break 
between  sets  of  trials.  AT  THE  END  OE  EACH  SET  OE  TRIALS,  YOU  WILL  BE  ASKED 
TO  COMPLETE  THE  NASA  TASK  LOAD  INDEX  QUESTIONNAIRE  in  which  you  will 
be  asked  to  rate  the  demands  of  the  task,  your  performance,  and  the  levels  of  effort  you 
exerted  and  frustration  you  experienced  during  the  task.  You  will  also  be  asked  to  eomplete 
the  SART  questionnaire  in  whieh  you  will  be  required  to  rate  other  dimensions  of  the  task. 
Those  will  be  deseribed  on  the  questionnaire.  Please  let  the  experimenter  know  when  a 
message  appears  on  the  sereen  indieating  that  it  is  time  to  fill  out  these  questionnaires. 

Please  use  your  LEPT  hand  for  the  YES  and  NO  keys  and  your  RIGHT  hand  to  seleet  with 
the  mouse. 
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Appendix  B2:  Instructions  for  Dyads 
Instructions  for  change  detection  task 


You  and  your  teammate  will  see  an  image  flashing  on  the  sereen  with  either  6  or  48 
colored  squares.  On  some  trials,  one  of  the  squares  will  ehange  in  position.  Your  job  is  to 
indieate  (1)  if  one  of  the  squares  changes  in  position  during  a  trial  and  (2)  whieh  square 
ehanged  in  position.  Please  press  “YES”  to  indicate  that  you  saw  one  of  the  squares  change 
in  position  and  “NO”  if  you  think  that  none  of  the  squares  changed  in  position.  On  trials  in 
which  you  respond  “YES,”  that  a  ehange  in  position  oecurred,  you  will  be  asked  to  indieate 
which  square  changed  in  position  by  using  the  mouse  to  select  it.  Please  respond  as  quiekly 
AND  accurately  as  you  ean.  To  continue  to  the  next  trial,  the  person  on  the  left  will  have  to 
press  the  spaee  bar. 

During  the  first  part  of  the  experiment,  you  and  your  teammate  will  not  be  allowed  to 
communicate  in  anyway  during  the  task  or  between  blocks  of  trials.  The  first  response  given 
during  a  trial  -  whether  from  you  or  your  teammate  -  will  be  the  reeorded  response  for  that 
trial.  During  the  second  part  of  the  experiment,  you  and  your  teammate  will  be  allowed  to 
communicate  during  the  experiment  and  are  encouraged  to  eome  up  with  a  strategy  that  will 
help  you  deteet  changes  in  the  images  on  the  sereen.  The  experimenter  will  indicate  the 
beginning  of  the  second  part  of  the  experiment. 

The  task  consists  of  2  sets  of  trials,  one  set  with  only  6  squares  on  the  screen  and  one 
set  with  48  squares  on  the  sereen.  The  number  of  squares  on  the  screen  will  not  vary  within  a 
set  of  trials.  You  will  start  with  either  one  or  the  other  set  and  you  will  be  given  a  break 
between  sets  of  trials.  AT  THE  END  OE  EACH  SET  OE  TRIALS,  YOU  WILL  BE  ASKED 


62 


TO  COMPLETE  THE  NASA  TASK  EOAD  INDEX  QUESTIONNAIRE  in  which  you  will 
be  asked  to  rate  the  demands  of  the  task,  your  performance,  and  the  levels  of  effort  you 
exerted  and  frustration  you  experienced  during  the  task.  You  will  also  be  asked  to  eomplete 
the  SITUATION  AWARENESS  QUESTIONNAIRE  and  the  TEAM  WORKLOAD  SCALE 
in  whieh  you  will  be  required  to  rate  other  dimensions  of  the  task.  Those  will  be  described 
on  the  questionnaires.  Please  let  the  experimenter  know  when  a  message  appears  on  the 
sereen  indicating  that  it  is  time  to  fill  out  these  questionnaires. 

Please  use  your  EEET  hand  for  the  YES  and  NO  keys  and  your  RIGHT  hand  to  select  with 
the  mouse. 
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Appendix  C:  Questionnaires 
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Appendix  Cl:  NASA  TLX 


Low 


Low 


Low 


Change  Blindness  Study 

Participant  I.D.; _ Date: _  Trial  #: 

Condition:. 


NASA  TLX  (Task  Load  Index)  Rating  Scales 

Instructions:  For  each  of  the  scales  presented  below,  please  write  the  number  that  matches 
your  experience  with  what  you  just  completed.  Please  note  that  the  "Performance"  scale 
goes  from  "good"  on  the  left  to  "poor"  on  the  right. 


Mental  Demand 

Performance 

1  1  1  1  1  1  1  1  1  1  |l  1  1  l|  1  1  1  1  1 

High 

Good 

1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1 

Poor 

0  25  50  75  100 

0  25  50  75  100 

Physical  Demand 

Effort 

1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1 

High 

Low 

1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1 

High 

0  25  50  75  100 

0  25  50  75  100 

Temporal  Demand 

Frustration 

1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1 

High 

Low 

1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1 

High 

0  25  50  75  100 

0  25  50  75  100 
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Appendix  C2:  Team  Work  Load  Scale 


Communication  Demand — 

How  much  communication  was  required  between  you  and  other  team  members  directly  or  via  the 
computer? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17 

Low 

Did  requesting  and  transferring  information  consume  a  little  of  your  time  or  a  lot? 

18  19  20 

High 

1  2  3  4  5  6  7  8  9  10  11  12  13  14 

Little 

Was  it  easy  or  demanding? 

15  16  17 

18  19  20  A 

A  Lot 

1  2  3  4  5  6  7  8  9  10  11  12  13  14 

Easy 

Monitoring  Demand — 

How  much  monitoring  of  people  did  the  task  require? 

15  16  17 

18  19  20 

Demanding 

1  2  3  4  5  6  7  8  9  10  11  12  13  14 

Low 

Was  attending  to  others  (directly  or  through  the  computer)  easy 

15  16  17 

or  demanding? 

18  19  20 

High 

1  2  3  4  5  6  7  8  9  10  11  12  13  14 

Easy 

15  16  17 

18  19  20 

Demanding 

Was  attending  to  others  (directly  or  through  the  computer)  infrequent  or  continuous? 

1  2  3  4  5  6  7  8  9  10  11  12  13  14 

Infrequent 

Control  Demand — 

How  much  correction  of  others  did  the  task  require? 

15  16  17 

18  19  20 

Continuous 

1  2  3  4  5  6  7  8  9  10  11  12  13  14 

Little 

Was  correcting  other  people  simple  or  complex? 

15  16  17 

18  19  20  A 

A  Lot 

1  2  3  4  5  6  7  8  9  10  11  12  13  14 

Simple 

15  16  17 

18  19  20 

Complex 
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Was  correcting  other  people  infrequent  or  continuous? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 

Infrequent  Continuous 

Coordination  Demand — 

How  much  correction  or  adjustment  of  your  own  actions  was  required  in  order  to  coordinate  with 

others? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 

A  Little  A  Lot 

Was  adjusting  your  actions  to  improve  coordination  simple  or  complex? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 

Simple  Complex 

Was  adjusting  your  actions  to  improve  coordination  infrequent  or  continuous? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 

Infrequent  Continuous 

Leadership  Demand — 

How  much  leadership  was  required  of  you  and/or  your  team  members? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 

A  Little  A  Lot 

Was  being  a  leader  easy  or  demanding? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 

Easy  Demanding 

Was  a  leader  needed  infrequently  or  frequently? 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 

Infrequently  Frequently 
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Appendix  C3:  Exit  Questionnaire 


Change  Blindness  Study 


Participant  I,D.: 


Date: 


Trial  #: 


Condition:. 


Exit  Questionnaire 


Did  you  adopt  any  strategies  to  aid  yourself  in  detecting  changes?  If  so,  what  were  they? 


At  any  point  throughout  the  experiment  did  you  have  a  feeling  that  something  was  changing 
but  could  not  identify  it? 
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When  a  change  was  noticed  did  you  respond  immediately  or  did  you  wait  for  another 
iteration  to  occur  so  you  could  see  the  change  again  before  responding? 


Did  you  notice  any  patterns  in  the  changes? 


How  confident  were  you  when  it  came  to  detecting  changes?  Please  circle  your  response  by 
using  a  1-10  rating  scale  (1:  not  confident,  10:  very  confident). 


123456789  10 

not  confident  very  confident 


Any  additional  comments? 
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Appendix  D:  Analysis  of  Accuracy  from  Individuals  and  Comm  Dyads  With  and 


Without  Participant  Q 
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INCLUDING  PARTICIPANT  Q’s  DATA 


Effects  coding  used  for  categorical  variables  in  model. 
Categorical  values  encountered  during  processing  are: 
indiv_v_comm  $  (2  levels) 
individuals,  comm  dyads 
7  case(s)  deleted  due  to  missing  data. 

Number  of  cases  processed:  16 
Dependent  variable  means 


AC_6LG 

AC_6SM 

AC_6NC 

AC_48LG 

AC_48SM 

AC_48NC 

0.995 

0.896 

0.995 

0.896 

0.708 

0.999 

Repeated  measures  factors  and  levels 
Dependent  Variables 


Within  factor 

1 

2 

3 

4 

5 

6 

set  size 

1.000 

1.000 

1.000 

2.000 

2.000 

2.000 

change  magn 

1.000 

2.000 

3.000 

1.000 

2.000 

3.000 

Univariate  and  multivariate  Repeated  Measures  Analysis 
_ Between  Subjects _ 


Source  SS 

df 

MS 

F 

P 

indiv  V  comm  $  0.196 

1 

0.196 

5.622 

0.033 

Error  0.488 

14 

0.035 

Within  Subjects 

Source  SS 

df 

MS 

F 

P  G-G 

H-F 

setsize  0.177 

1 

0.177 

9.780 

0.007 

setsize*  indiv  v  comm$  0.081 

1 

0.081 

4.448 

0.053 

Error  0.254 

14 

0.018 

Greenhouse-Geisser  Epsilon: 
Huynh-Feldt  Epsilon  : 


Source 

SS 

df 

MS 

F 

P 

G-G 

H-F 

changemagn 

0.552 

2 

0.276 

17.431 

0.000 

0.000 

0.000 

changemagn*  indiv  v  comm  $ 

0.218 

2 

0.109 

6.875 

0.004 

0.008 

0.005 

Error 

0.444 

28 

0.016 

Greenhouse-Geisser  Epsilon:  0.7713 

Huynh-Feldt  Epsilon  :  0.9103 


Source 

SS 

df 

MS 

F 

P 

G-G 

H-F 

setsize  *changemagn 

0.122 

2 

0.061 

5.538 

0.009 

0.012 

0.009 

setsize*changemagn*  indiv  v  comm  $ 

0.066 

2 

0.033 

2.999 

0.066 

0.071 

0.066 

Error 

0.309 

28 

0.011 

Greenhouse-Geisser  Epsilon:  0.9197 

Huynh-Feldt  Epsilon  :  1.0000 

Multivariate  repeated  measures  analysis 
Test  of:  changemagn 
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Statistic 

Value 

Hypoth.  df 

Error  df 

F 

P 

Wilks'  Lambda 

0.381 

2 

13 

10.548 

0.002 

Pillai  Trace 

0.619 

2 

13 

10.548 

0.002 

H-L  Trace 

1.623 

2 

13 

10.548 

0.002 

Test  of:  changemagn*  indiv  v  comm  $ 

Statistic 

Value 

Hypoth.  df 

Error  df 

F 

P 

Wilks'  Lambda 

0.600 

2 

13 

4.340 

0.036 

Pillai  Trace 

0.400 

2 

13 

4.340 

0.036 

H-L  Trace 

0.668 

2 

13 

4.340 

0.036 

Test  of:  setsize*changemagn 

Statistic  Value 

Hypoth.  df 

Error  df 

F 

P 

Wilks'  Lambda 

0.509 

2 

13 

6.258 

0.012 

Pillai  Trace 

0.491 

2 

13 

6.258 

0.012 

H-L  Trace 

0.963 

2 

13 

6.258 

0.012 

Test  of:  setsize*changemagn* 

indiv  V  comm  $ 

Statistic 

Value 

Hypoth.  df 

Error  df 

F 

P 

Wilks'  Lambda 

0.667 

2 

13 

3.248 

0.072 

Pillai  Trace 

0.333 

2 

13 

3.248 

0.072 

H-L  Trace 

0.500 

2 

13 

3.248 

0.072 
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EXCLUDING  PARTICIPANT  Q’s  DATA 

Effects  coding  used  for  categorical  variables  in  model. 
Categorical  values  encountered  during  processing  are: 
indiv  v  comm  $  (2  levels) 
individuals,  comm  dyads 
7  case(s)  deleted  due  to  missing  data. 

Number  of  cases  processed:  15 
Dependent  variable  means 


AC  6LG  AC  6SM 

AC  6NC  AC  48LG 

AC  48  SM 

AC  48NC 

0.994  0.917 

0.994  0.939 

0.750 

0.999 

Repeated  measures  factors  and  levels 
Dependent  Variables 

Within  factor 

1  2 

3 

4 

5 

6 

set  size 

1.000  1.000 

1.000 

2.000 

2.000 

2.000 

change  magn 

1.000  2.000 

3.000 

1.000 

2.000 

3.000 

Univariate  and  multivariate  Repeated  Measures  Analysis 

Between  Subjects 

Source  SS 

df  MS 

F 

P 

indiv  V  comm  $  0.090 

1  0.090 

7.551 

0.017 

Error  0.154 

13  0.012 

Within  Subjects 

Source 

SS  df 

MS 

F 

P 

G-G 

H-F 

setsize 

0.109  1 

0.109 

10.906 

0.006 

setsize*  indiv  v  comm  $ 

0.039  1 

0.039 

3.863 

0.071 

Error 

0.130  13 

0.010 

Greenhouse-Geisser  Epsilon: 

Huynh-Feldt  Epsilon  : 

Source  SS  df 

MS 

F 

P 

G-G 

H-F 

changemagn  0.418  2 

0.209 

20.581 

0.000 

0.000 

0.000 

changemagn*  indiv  v  comm  $  0.149  2 

0.074 

7.321 

0.003 

0.014 

0.011 

Error  0.264  26 

0.010 

Greenhouse-Geisser  Epsilon: 

0.5732 

Huynh-Feldt  Epsilon  :  ( 

0.6409 

Source  SS  df 

MS 

F 

P 

G-G 

H-F 

setsize*changemagn  0.103  2 

0.051 

6.476 

0.005 

0.017 

0.013 

setsize*changemagn*  indiv  v  comm  $  0.057  2 

0.029 

3.614 

0.041 

0.068 

0.061 

Error  0.206  26 

0.008 

Greenhouse-Geisser  Epsilon:  0.6234 

Huynh-Feldt  Epsilon  :  0.7105 

Multivariate  repeated  measures  analysis 
Test  of:  changemagn 
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Statistic 

Value  Hypoth.  df  Error  df 

F 

P 

Wilks'  Lambda 

0.247 

2  12 

18.258 

0.000 

Pillai  Trace 

0.753 

2  12 

18.258 

0.000 

H-L  Trace 

3.043 

2  12 

18.258 

0.000 

Test  of:  changemagn*  indiv  v  comm  $ 

Statistic  Value 

Hypoth.  df 

Error  df 

F 

P 

Wilks'  Lambda  0.603 

2 

12 

3.946 

Pillai  Trace  0.397 

2 

12 

3.946 

H-L  Trace  0.658 

2 

12 

3.946 

Test  of:  setsize*changemagn 

Statistic  Value 

Hypoth.  df 

Error  df 

F 

p 

Wilks'  Lambda  0.437 

2 

12 

7.725 

0.007 

Pillai  Trace  0.563 

2 

12 

7.725 

0.007 

H-L  Trace  1.287 

2 

12 

7.725 

0.007 

Test  of:  setsize*changemagn*  indiv_v_comm  $ 


Statistic 

Value 

Hypoth.  df 

Error  df 

F 

P 

Wilks'  Lambda 

0.675 

2 

12 

2.884 

0.095 

Pillai  Trace 

0.325 

2 

12 

2.884 

0.095 

H-L  Trace 

0.481 

2 

12 

2.884 

22  cases  and  50  variables  processed  and  saved. 
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Appendix  E:  Data 
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Appendix  El:  Performance  and  fixations  data,  dyads 
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Appendix  E2:  Performance  and  fixations  data,  Individuals  V5  Dyads 
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Appendix  E3:  Rescanning  and  crossscanning  data,  Dyads 
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Appendix  E4:  Rescanning  and  crossscanning  data,  Individuals  V5  Dyads 


Group$ 

ivnocom$ 

ivcom$ 

renochd 

re_smch6 

re_lgch6 

re_noch48 

re_smch48 

re_lgch48 

indiv_2 

indiv 

indiv 

0.917 

0.727 

0.833 

1.000 

1.000 

1.000 

indiv_3 

indiv 

indiv 

1.000 

1.000 

0.833 

1.000 

1.000 

0.917 

indiv_7 

indiv 

indiv 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

indivlO 

indiv 

indiv 

1.000 

1.000 

1.000 

1.000 

1.000 

0.833 

indiv_14 

indiv 

indiv 

0.667 

0.600 

0.917 

1.000 

1.000 

1.000 

indivl  5 

indiv 

indiv 

0.909 

0.833 

0.917 

1.000 

1.000 

1.000 

indivl  6 

indiv 

indiv 

0.917 

0.750 

0.583 

1.000 

0.818 

1.000 

indiv_20 

indiv 

indiv 

1.000 

1.000 

0.917 

1.000 

1.000 

1.000 

indiv_2 1 

indiv 

indiv 

0.667 

0.714 

0.583 

1.000 

1.000 

1.000 

dyad_3_l 

no  Comm 

1.000 

1.000 

0.833 

1.000 

1.000 

0.900 

dyad_4_l 

no  Comm 

1.000 

0.667 

0.583 

1.000 

1.000 

1.000 

dyad_5_l 

no  Comm 

0.333 

0.250 

0.364 

1.000 

1.000 

0.714 

dyad_6_l 

no  Comm 

1.000 

0.750 

0.583 

1.000 

0.909 

0.917 

dyad_8_l 

no  Comm 

1.000 

0.917 

0.750 

1.000 

1.000 

1.000 

dyad_9_l 

no  Comm 

0.833 

0.833 

0.667 

1.000 

0.889 

0.917 

dyadl 11 

no  Comm 

0.818 

0.800 

0.917 

1.000 

1.000 

1.000 

dyad_3_2 

no  Comm 

1.000 

0.667 

0.750 

1.000 

0.917 

1.000 

dyad_4_2 

no  Comm 

0.833 

0.750 

0.667 

1.000 

1.000 

0.917 

dyad_5_2 

no  Comm 

0.917 

0.875 

0.727 

1.000 

1.000 

0.857 

dyad_6_2 

no  Comm 

0.917 

0.750 

0.583 

1.000 

0.909 

0.833 

dyad_8_2 

no  Comm 

0.917 

0.917 

0.583 

1.000 

1.000 

0.909 

dyad_9_2 

no  Comm 

0.833 

0.583 

0.333 

1.000 

0.778 

0.750 

dyad_l 1_2 

no  Comm 

0.818 

0.600 

0.500 

1.000 

1.000 

1.000 

dyad_3_l 

Comm 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

dyad_4_l 

Comm 

1.000 

0.917 

0.727 

1.000 

1.000 

1.000 

dyad_5_l 

Comm 

0.833 

0.727 

0.333 

0.917 

0.778 

1.000 

dyad_6_l 

Comm 

1.000 

0.818 

0.833 

1.000 

1.000 

0.917 

dyad_8_l 

Comm 

1.000 

0.917 

1.000 

1.000 

0.833 

1.000 

dyad_9_l 

Comm 

1.000 

0.667 

0.667 

1.000 

0.818 

0.818 

dyadl 11 

Comm 

1.000 

0.900 

0.750 

1.000 

1.000 

0.909 

dyad_3_2 

Comm 

0.917 

0.917 

0.750 

1.000 

1.000 

0.917 

dyad_4_2 

Comm 

1.000 

0.917 

0.818 

1.000 

0.917 

0.750 

dyad_5_2 

Comm 

1.000 

0.909 

0.667 

1.000 

1.000 

1.000 

dyad_6_2 

Comm 

1.000 

0.364 

0.750 

1.000 

1.000 

0.917 

dyad_8_2 

Comm 

0.917 

0.583 

0.500 

1.000 

0.833 

0.833 

dyad_9_2 

Comm 

0.833 

0.583 

0.500 

1.000 

0.909 

0.636 

dyad_l 1_2 

Comm 

0.833 

0.600 

0.750 

1.000 

0.900 

1.000 
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Appendix  E5:  workload  data,  Dyads 
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Appendix  E6:  workload  data,  Individuals  vs  Dyads 
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Appendix  E7:  NNI  correlations  to  heart  rate  and  RTLX 


SUBJECTS 

SETSIZES 

SUBJECT 

SETSIZE 

HR 

RTLX 

NNI 

indiv  2 

6.000 

1 

1 

35 

0.8490266 

indiv  3 

6.000 

1 

1 

35 

0.7980111 

indiv  7 

6.000 

1 

1 

20.833333 

0.8412377 

indiv  1 0 

6.000 

1 

1 

24.166667 

0.8741777 

indiv  14 

6.000 

1 

1 

711.325 

37.5 

0.916476 

indiv  15 

6.000 

1 

1 

805.00725 

33.333333 

0.7613795 

indiv  16 

6.000 

1 

1 

1094.3873 

23.166667 

0.9535818 

indiv  20 

6.000 

1 

1 

995.10427 

32.5 

0.8133558 

indiv  21 

6.000 

1 

1 

931.01575 

33.333333 

0.8990996 

dyad3  In 

6.000 

2 

1 

791.17452 

46.666667 

0.840472 

dyad4  In 

6.000 

2 

1 

858.6902 

23.333333 

0.7529475 

dyad5  In 

6.000 

2 

1 

909.31169 

60 

0.8183159 

dyad6  In 

6.000 

2 

1 

819.10781 

24.166667 

0.8208879 

dyadS  In 

6.000 

2 

1 

906.47603 

35.833333 

0.8659857 

dyad9  In 

6.000 

2 

1 

1186.1077 

31.333333 

0.8880612 

dyad 11  In 

6.000 

2 

1 

979.86194 

47.5 

0.7996474 

dyad3_2n 

6.000 

2 

1 

707.67901 

44.166667 

0.8272924 

dyad4  2n 

6.000 

2 

1 

715.47319 

52.5 

0.9552916 

dyad5_2n 

6.000 

2 

1 

1192.7787 

53.333333 

0.9003962 

dyad6  2n 

6.000 

2 

1 

727.83828 

20 

0.8574745 

dyadS  2n 

6.000 

2 

1 

787.02663 

21.666667 

0.8836778 

dyad9_2n 

6.000 

2 

1 

653.17839 

40 

0.7882323 

dyadl l_2n 

6.000 

2 

1 

835.01987 

57.5 

1.024145 

dyad3_lc 

6.000 

3 

1 

796.20755 

46.666667 

0.834237 

dyad4  1 c 

6.000 

3 

1 

856.22963 

21.666667 

0.8014305 

dyads  1 c 

6.000 

3 

1 

823.72911 

40 

0.8503429 

dyad6  Ic 

6.000 

3 

1 

841.94574 

22.5 

0.7449266 

dyad8_lc 

6.000 

3 

1 

846.79433 

25.833333 

0.8908347 

dyad9  Ic 

6.000 

3 

1 

1306.1859 

28.333333 

0.8828942 

dyad 11  1 c 

6.000 

3 

1 

941.66791 

36.666667 

0.864321 

dyad3_2c 

6.000 

3 

1 

696.42582 

25 

0.8320357 

dyad4  2c 

6.000 

3 

1 

747.22006 

21.666667 

0.8832723 

dyad5_2c 

6.000 

3 

1 

1055.3074 

33.333333 

0.8336229 

dyad6  2c 

6.000 

3 

1 

752.87847 

20 

0.8522809 

dyad8_2c 

6.000 

3 

1 

775.12945 

20 

0.8268384 

dyad9_2c 

6.000 

3 

1 

592.79744 

28.333333 

0.8376425 

dyadl l_2c 

6.000 

3 

1 

828.63722 

63.333333 

0.957224 

indiv  2 

48.000 

1 

2 

60.833333 

0.834775 

indiv  3 

48.000 

1 

2 

46.666667 

0.8823834 

indiv  7 

48.000 

1 

2 

60.833333 

0.8791435 

indiv  10 

48.000 

1 

2 

51.666667 

0.8961612 

indiv  14 

48.000 

1 

2 

730.32056 

55 

0.8175046 

indiv  1 5 

48.000 

1 

2 

805.99267 

57.5 

0.8792771 

indiv  16 

48.000 

1 

2 

1018.2524 

18.666667 

0.9159006 

indiv  20 

48.000 

1 

2 

891.65825 

32.5 

0.8133558 

indiv  21 

48.000 

1 

2 

924.96273 

25.833333 

0.9133735 

dyad3  In 

48.000 

2 

2 

877.23383 

49.166667 

0.8947366 

dyad4  In 

48.000 

2 

2 

830.08537 

43.333333 

0.8008798 

dyads  In 

48.000 

2 

2 

898.69048 

46.666667 

0.8251116 

dyad6  In 

48.000 

2 

2 

832.87302 

38.333333 

0.8924323 

dyadS  In 

48.000 

2 

2 

851.64796 

45 

0.8250864 

dyad9  In 

48.000 

2 

2 

1104.326 

28.833333 

0.8250864 

dyad 11  In 

48.000 

2 

2 

996.98675 

55 

0.8857912 

dyad3_2n 

48.000 

2 

2 

716.00947 

63.333333 

0.849566 

dyad4  2n 

48.000 

2 

2 

715.98494 

45.833333 

0.9528295 

dyad5_2n 

48.000 

2 

2 

1046.0886 

63.333333 

0.9282681 

dyad6  2n 

48.000 

2 

2 

732.01198 

34.166667 

0.7422238 

dyadS  2n 

48.000 

2 

2 

776.0031 

25 

0.9943563 

dyad9  2n 

48.000 

2 

2 

613.87942 

41.666667 

0.8251701 

dyadl l_2n 

48.000 

2 

2 

830.99119 

58.333333 

0.8691516 

dyad3_lc 

48.000 

3 

2 

823.75147 

48.333333 

0.8391671 

dyad4  1 c 

48.000 

3 

2 

847.16507 

46.666667 

0.8577682 

dyads  1 c 

48.000 

3 

2 

825.975 

35.833333 

0.7840848 

dyad6  Ic 

48.000 

3 

2 

834.37132 

30.833333 

0.7932671 

dyad8_lc 

48.000 

3 

2 

870.04936 

31.666667 

0.8410208 

dyad9  Ic 

48.000 

3 

2 

1276.1433 

20.333333 

0.8393137 

dyad 11  1 c 

48.000 

3 

2 

903.29426 

54.166667 

0.7783499 

dyad3_2c 

48.000 

3 

2 

688.61866 

48.333333 

0.6282463 

dyad4  2c 

48.000 

3 

2 

746.87648 

46.666667 

0.8536838 

dyads  2c 

48.000 

3 

2 

916.12911 

53.333333 

0.6798736 

dyad6_2c 

48.000 

3 

2 

736.96707 

22.5 

0.7791402 

dyad8_2c 

48.000 

3 

2 

776.82871 

20 

0.9109248 

dyad9_2c 

48.000 

3 

2 

622.38728 

38.333333 

0.8136616 

dyadl l_2c 

48.000 

3 

2 

826.01828 

65 

0.7599139 

84 


Appendix  E8:  Regression  model  of  change  detection  with  multiple  observers 
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